Attachment 1454319 Details for Bug 1594698 – volume.log from the controller

volume.log from the controller

volume.log (text/plain), 11.05 MB, created by Ronnie Rasouli on 2018-06-25 10:50:42 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Ronnie Rasouli

Created: 2018-06-25 10:50:42 UTC

Size: 11.05 MB

patch

obsolete

>2018-06-24 13:00:24.220 139116 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
>2018-06-24 13:00:24.221 139116 DEBUG oslo_concurrency.lockutils [-] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-24 13:00:24.221 139116 DEBUG oslo_concurrency.lockutils [-] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-24 13:00:24.611 139116 DEBUG oslo_db.sqlalchemy.engines [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-24 13:00:24.766 139116 INFO cinder.rpc [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version.
>2018-06-24 13:00:24.771 139116 INFO cinder.rpc [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version.
>2018-06-24 13:00:24.778 139116 DEBUG oslo_db.sqlalchemy.engines [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-24 13:00:24.786 139116 INFO cinder.volume.manager [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Service not found for updating active_backend_id, assuming default for driver init.
>2018-06-24 13:00:24.787 139116 WARNING cinder.keymgr.conf_key_mgr [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] This key manager is insecure and is not recommended for production deployments: ServiceNotFound: Service cinder-volume could not be found on host hostgroup@tripleo_ceph.
>2018-06-24 13:00:24.810 139116 INFO cinder.volume.manager [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Image-volume cache disabled for host hostgroup@tripleo_ceph.
>2018-06-24 13:00:24.816 139116 DEBUG oslo_db.sqlalchemy.engines [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-24 13:00:24.829 139116 INFO oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Starting 1 workers
>2018-06-24 13:00:24.834 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Started child 139328 _start_child /usr/lib/python2.7/site-packages/oslo_service/service.py:513
>2018-06-24 13:00:24.840 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Full set of CONF: wait /usr/lib/python2.7/site-packages/oslo_service/service.py:580
>2018-06-24 13:00:24.841 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2865
>2018-06-24 13:00:24.841 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Configuration options gathered from: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2866
>2018-06-24 13:00:24.841 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] command line args: ['--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--logfile', '/var/log/cinder/volume.log'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2867
>2018-06-24 13:00:24.842 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] config files: ['/usr/share/cinder/cinder-dist.conf', '/etc/cinder/cinder.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2869
>2018-06-24 13:00:24.842 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] ================================================================================ log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2870
>2018-06-24 13:00:24.843 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] allow_availability_zone_fallback = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.842 139328 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0)
>2018-06-24 13:00:24.843 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] allowed_direct_url_schemes     = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.844 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] api_rate_limit                 = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.844 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] auth_strategy                  = keystone log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.845 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_availability_zone      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.845 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_host                   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.847 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backup_api_class               = cinder.backup.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.848 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backup_manager                 = cinder.backup.manager.BackupManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.849 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backup_name_template           = backup-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.849 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backup_use_same_host           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.849 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backup_use_temp_snapshot       = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.850 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] chap_password                  = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.850 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] chap_username                  =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.851 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.852 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] cinder_internal_tenant_project_id = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.852 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] cinder_internal_tenant_user_id = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.852 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] cluster                        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.853 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] compute_api_class              = cinder.compute.nova.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.853 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] config_dir                     = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.854 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] config_file                    = ['/usr/share/cinder/cinder-dist.conf', '/etc/cinder/cinder.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.854 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] consistencygroup_api_class     = cinder.consistencygroup.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.855 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] control_exchange               = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.855 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] db_driver                      = cinder.db log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.856 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] debug                          = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.856 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] default_availability_zone      = nova log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.857 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] default_group_type             = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.858 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'dogpile.core.dogpile=INFO'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.858 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] default_volume_type            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.858 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] driver_client_cert             = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.859 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] driver_client_cert_key         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.859 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] driver_data_namespace          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.860 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] driver_ssl_cert_path           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.860 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] driver_ssl_cert_verify         = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.861 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] driver_use_ssl                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.861 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] enable_new_services            = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.861 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] enable_unsupported_driver      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.862 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] enable_v1_api                  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.862 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] enable_v2_api                  = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.863 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] enable_v3_api                  = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.863 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] enabled_backends               = ['tripleo_ceph'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.863 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] enforce_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.864 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] fatal_exception_format_errors  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.864 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] filter_function                = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.865 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] glance_api_insecure            = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.866 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] glance_api_servers             = ['http://172.17.1.12:9292'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.866 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] glance_api_ssl_compression     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.867 139116 WARNING oslo_config.cfg [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens).  Its value may be silently ignored in the future.
>2018-06-24 13:00:24.868 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] glance_api_version             = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.868 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] glance_ca_certificates_file    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.869 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] glance_catalog_info            = image:glance:publicURL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.869 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.870 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] glance_num_retries             = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.870 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] glance_request_timeout         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.870 139328 INFO cinder.volume.manager [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Including all resources from host hostgroup@tripleo_ceph in cluster None.
>2018-06-24 13:00:24.871 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] goodness_function              = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.872 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] graceful_shutdown_timeout      = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.872 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] group_api_class                = cinder.group.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.874 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] host                           = hostgroup log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.875 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iet_conf                       = /etc/iet/ietd.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.876 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] image_conversion_dir           = /var/lib/cinder/conversion log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.877 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] image_upload_use_cinder_backend = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.877 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] image_upload_use_internal_tenant = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.878 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] image_volume_cache_enabled     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.878 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] image_volume_cache_max_count   = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.879 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] image_volume_cache_max_size_gb = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.879 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] instance_format                = [instance: %(uuid)s]  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.880 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] instance_uuid_format           = [instance: %(uuid)s]  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.880 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iscsi_helper                   = lioadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.881 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iscsi_iotype                   = fileio log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.882 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iscsi_ip_address               = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.882 139328 DEBUG oslo_db.sqlalchemy.engines [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-24 13:00:24.882 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iscsi_port                     = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.883 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iscsi_protocol                 = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.884 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iscsi_secondary_ip_addresses   = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.884 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iscsi_target_flags             =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.885 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iscsi_target_prefix            = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.885 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iscsi_write_cache              = on log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.886 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iser_helper                    = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.887 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iser_ip_address                = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.887 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iser_port                      = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.888 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] iser_target_prefix             = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.888 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] log_config_append              = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.889 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] log_date_format                = %Y-%m-%d %H:%M:%S log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.890 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] log_dir                        = /var/log/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.891 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] log_file                       = /var/log/cinder/volume.log log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.891 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] log_options                    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.892 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] logging_context_format_string  = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.892 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] logging_debug_format_suffix    = %(funcName)s %(pathname)s:%(lineno)d log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.893 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] logging_default_format_string  = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.893 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] logging_exception_prefix       = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.894 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] logging_user_identity_format   = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.894 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] max_age                        = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.895 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] max_over_subscription_ratio    = 20.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.895 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] message_reap_interval          = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.896 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] message_ttl                    = 2592000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.896 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] migration_create_volume_timeout_secs = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.897 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] monkey_patch                   = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.897 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] monkey_patch_modules           = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.898 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] my_ip                          = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.898 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] no_snapshot_gb_quota           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.898 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] num_iser_scan_tries            = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.899 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] num_shell_tries                = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.899 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] num_volume_device_scan_tries   = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.899 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] os_privileged_user_auth_url    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.900 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] os_privileged_user_name        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.900 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] os_privileged_user_password    = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.901 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] os_privileged_user_tenant      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.901 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] osapi_max_limit                = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.902 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] osapi_volume_base_URL          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.902 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] osapi_volume_ext_list          = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.902 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.903 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] osapi_volume_listen            = controller-0.internalapi.localdomain log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.903 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] osapi_volume_listen_port       = 8776 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.903 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] osapi_volume_use_ssl           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.904 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] osapi_volume_workers           = 4 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.904 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] per_volume_size_limit          = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.905 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] periodic_fuzzy_delay           = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.905 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] periodic_interval              = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.905 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] public_endpoint                = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.905 139328 INFO cinder.volume.manager [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] 0 volumes, 0 consistency groups, 0 generic groups and 0 image volume caches from host hostgroup@tripleo_ceph have been included in cluster None.
>2018-06-24 13:00:24.906 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] publish_errors                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.906 139328 INFO cinder.volume.manager [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Starting volume driver RBDDriver (1.2.0)
>2018-06-24 13:00:24.906 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] query_volume_filters           = ['name', 'status', 'metadata', 'availability_zone', 'bootable', 'group_id'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.906 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] quota_backup_gigabytes         = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.907 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] quota_backups                  = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.907 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] quota_consistencygroups        = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.907 139328 DEBUG cinder.volume.drivers.rbd [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:00:24.908 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] quota_driver                   = cinder.quota.DbQuotaDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.908 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] quota_gigabytes                = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.908 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] quota_groups                   = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.909 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] quota_snapshots                = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.909 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] quota_volumes                  = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.909 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] rate_limit_burst               = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.910 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] rate_limit_except_level        = CRITICAL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.910 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] rate_limit_interval            = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.911 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] replication_device             = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.911 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] report_discard_supported       = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.912 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] report_interval                = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.912 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] reservation_clean_interval     = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.912 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] reservation_expire             = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.913 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] reserved_percentage            = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.913 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] resource_query_filters_file    = /etc/cinder/resource_filters.json log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.913 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] rootwrap_config                = /etc/cinder/rootwrap.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.914 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] rpc_backend                    = rabbit log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.914 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] rpc_response_timeout           = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.915 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] run_external_periodic_tasks    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.915 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] scheduler_manager              = cinder.scheduler.manager.SchedulerManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.915 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] scst_target_driver             = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.916 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] scst_target_iqn_name           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.916 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] service_down_time              = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.916 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] snapshot_name_template         = snapshot-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.917 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] state_path                     = /var/lib/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.917 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] storage_availability_zone      = nova log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.917 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] storage_protocol               = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.918 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] syslog_log_facility            = LOG_USER log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.918 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] trace_flags                    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.919 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] transfer_api_class             = cinder.transfer.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.919 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] transport_url                  = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.919 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] until_refresh                  = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.920 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] use_chap_auth                  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.920 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] use_default_quota_class        = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.921 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] use_journal                    = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.921 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] use_multipath_for_image_xfer   = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.921 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] use_stderr                     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.922 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] use_syslog                     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.922 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_api_class               = cinder.volume.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.923 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_backend_name            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.923 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_clear                   = zero log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.924 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_clear_ionice            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.924 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_clear_size              = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.925 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_copy_blkio_cgroup_name  = cinder-volume-copy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.925 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_copy_bps_limit          = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.926 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_dd_blocksize            = 1M log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.926 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_manager                 = cinder.volume.manager.VolumeManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.926 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_name_template           = volume-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.927 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_service_inithost_offload = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.927 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volume_usage_audit_period      = month log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.928 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] volumes_dir                    = /etc/cinder/volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.928 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] watch_log_file                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.928 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] zoning_mode                    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-24 13:00:24.929 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.amqp_auto_delete = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.930 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.amqp_durable_queues = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.930 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.conn_pool_min_size = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.930 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.conn_pool_ttl = 1200 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.931 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.fake_rabbit = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.931 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.heartbeat_rate = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.932 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.932 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.kombu_compression = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.933 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.kombu_failover_strategy = round-robin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.933 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.934 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.935 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_ha_queues = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.935 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_host = localhost log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.936 139116 WARNING oslo_config.cfg [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-24 13:00:24.936 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_hosts = ['localhost:5672'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.937 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_interval_max = 30 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.937 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.938 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_max_retries = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.938 139116 WARNING oslo_config.cfg [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-24 13:00:24.938 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.939 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_port = 5672 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.939 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.939 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_retry_backoff = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.940 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_retry_interval = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.940 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.941 139116 WARNING oslo_config.cfg [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-24 13:00:24.941 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_userid = guest log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.941 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rabbit_virtual_host = / log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.942 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.rpc_conn_pool_size = 30 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.943 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.ssl      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.943 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.ssl_ca_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.944 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.ssl_cert_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.945 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.ssl_key_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.945 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_rabbit.ssl_version =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.946 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] coordination.backend_url       = file:///var/lib/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.947 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] coordination.heartbeat         = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.947 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] coordination.initial_reconnect_backoff = 0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.948 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] coordination.max_reconnect_backoff = 60.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.949 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.backend               = sqlalchemy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.949 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.connection            = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.950 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.connection_debug      = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.950 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.connection_trace      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.951 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.db_inc_retry_interval = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.951 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.db_max_retries        = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.951 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.db_max_retry_interval = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.952 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.db_retry_interval     = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.953 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.idle_timeout          = 3600 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.953 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.max_overflow          = 50 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.954 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.max_pool_size         = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.954 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.max_retries           = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.955 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.min_pool_size         = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.955 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.mysql_enable_ndb      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.955 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.mysql_sql_mode        = TRADITIONAL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.956 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.pool_timeout          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.956 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.retry_interval        = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.957 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.slave_connection      = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.957 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.sqlite_synchronous    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.958 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] database.use_db_reconnect      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.959 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_notifications.driver = ['messagingv2'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.959 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_notifications.retry = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.959 139328 DEBUG cinder.volume.drivers.rbd [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:00:24.960 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_notifications.topics = ['notifications'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.960 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_messaging_notifications.transport_url = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.961 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_reports.file_event_handler = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.961 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_reports.file_event_handler_interval = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.961 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_reports.log_dir           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.962 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.backend_availability_zone = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.962 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.backend_host      = hostgroup log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.963 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.backup_use_temp_snapshot = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.963 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.chap_password     = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.963 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.chap_username     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.964 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.chiscsi_conf      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.964 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.driver_client_cert = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.965 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.driver_client_cert_key = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.965 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.driver_data_namespace = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.965 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.driver_ssl_cert_path = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.966 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.driver_ssl_cert_verify = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.966 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.driver_use_ssl    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.966 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.enable_unsupported_driver = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.967 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.enforce_multipath_for_image_xfer = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.967 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.extra_capabilities = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.967 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.filter_function   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.968 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.goodness_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.968 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iet_conf          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.969 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.image_upload_use_cinder_backend = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.969 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.image_upload_use_internal_tenant = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.969 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.image_volume_cache_enabled = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.970 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.image_volume_cache_max_count = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.970 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.image_volume_cache_max_size_gb = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.971 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iscsi_helper      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.971 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iscsi_iotype      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.971 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iscsi_ip_address  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.972 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iscsi_port        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.972 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iscsi_protocol    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.973 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iscsi_secondary_ip_addresses = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.973 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iscsi_target_flags = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.974 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iscsi_target_prefix = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.974 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iscsi_write_cache = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.975 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iser_helper       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.975 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iser_ip_address   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.975 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iser_port         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.975 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.iser_target_prefix = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.976 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.max_over_subscription_ratio = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.976 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.num_iser_scan_tries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.976 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.num_shell_tries   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.977 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.num_volume_device_scan_tries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.977 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rados_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.978 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rados_connection_interval = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.978 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rados_connection_retries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.978 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rbd_ceph_conf     = /etc/ceph/ceph.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.979 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rbd_cluster_name  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.979 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rbd_flatten_volume_from_snapshot = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.980 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rbd_keyring_conf  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.980 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rbd_max_clone_depth = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.981 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rbd_pool          = volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.981 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rbd_secret_uuid   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.981 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rbd_store_chunk_size = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.982 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.rbd_user          = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.982 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.replication_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.982 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.replication_device = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.983 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.report_discard_supported = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.983 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.report_dynamic_total_capacity = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.984 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.reserved_percentage = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.984 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.scst_target_driver = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.984 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.scst_target_iqn_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.985 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.storage_protocol  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.985 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.suppress_requests_ssl_warnings = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.986 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.trace_flags       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.987 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.use_chap_auth     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.987 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.use_multipath_for_image_xfer = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.988 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.volume_backend_name = tripleo_ceph log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.988 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.volume_clear      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.989 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.volume_clear_ionice = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.990 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.volume_clear_size = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.990 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.volume_copy_blkio_cgroup_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.991 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.volume_copy_bps_limit = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.991 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.volume_dd_blocksize = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.992 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.volume_driver     = cinder.volume.drivers.rbd.RBDDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.992 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] tripleo_ceph.volumes_dir       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.993 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] profiler.connection_string     = messaging:// log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.993 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] profiler.enabled               = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.994 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] profiler.es_doc_type           = notification log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.994 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] profiler.es_scroll_size        = 10000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.994 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] profiler.es_scroll_time        = 2m log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.995 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] profiler.hmac_keys             = SECRET_KEY log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.996 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] profiler.sentinel_service_name = mymaster log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.996 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] profiler.socket_timeout        = 0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.997 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] profiler.trace_sqlalchemy      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.997 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] privsep_osbrick.capabilities   = [21] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.998 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] privsep_osbrick.group          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.998 139328 DEBUG cinder.volume.drivers.rbd [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:00:24.999 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] privsep_osbrick.helper_command = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:24.999 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] privsep_osbrick.user           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.000 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.backend_availability_zone = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.000 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.backup_use_temp_snapshot = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.001 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.chap_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.002 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.chap_username =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.002 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.003 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.driver_client_cert = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.003 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.driver_client_cert_key = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.003 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.driver_data_namespace = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.004 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.driver_ssl_cert_path = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.004 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.driver_ssl_cert_verify = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.004 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.driver_use_ssl = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.005 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.enable_unsupported_driver = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.005 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.enforce_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.005 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.extra_capabilities = {} log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.006 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.filter_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.006 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.goodness_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.006 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iet_conf      = /etc/iet/ietd.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.007 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.image_upload_use_cinder_backend = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.007 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.image_upload_use_internal_tenant = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.007 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.image_volume_cache_enabled = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.008 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.image_volume_cache_max_count = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.008 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.image_volume_cache_max_size_gb = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.008 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iscsi_helper  = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.009 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iscsi_iotype  = fileio log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.009 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iscsi_ip_address = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.009 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iscsi_port    = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.010 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iscsi_protocol = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.010 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iscsi_secondary_ip_addresses = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.010 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iscsi_target_flags =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.011 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iscsi_target_prefix = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.011 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iscsi_write_cache = on log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.011 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iser_helper   = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.012 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iser_ip_address = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.012 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iser_port     = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.013 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.013 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.max_over_subscription_ratio = 20.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.013 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.num_iser_scan_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.014 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.num_shell_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.014 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.num_volume_device_scan_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.014 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rados_connect_timeout = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.015 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rados_connection_interval = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.015 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rados_connection_retries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.015 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rbd_ceph_conf =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.015 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rbd_cluster_name = ceph log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.016 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rbd_flatten_volume_from_snapshot = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.016 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rbd_keyring_conf =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.016 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rbd_max_clone_depth = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.017 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rbd_pool      = rbd log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.017 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rbd_secret_uuid = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.017 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rbd_store_chunk_size = 4 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.018 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.rbd_user      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.018 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.replication_connect_timeout = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.018 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.replication_device = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.019 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.report_discard_supported = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.019 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.report_dynamic_total_capacity = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.019 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.reserved_percentage = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.020 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.scst_target_driver = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.020 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.scst_target_iqn_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.020 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.storage_protocol = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.021 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.suppress_requests_ssl_warnings = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.021 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.trace_flags   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.021 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.use_chap_auth = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.022 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.use_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.022 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.volume_backend_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.022 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.volume_clear  = zero log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.023 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.volume_clear_ionice = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.023 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.volume_clear_size = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.023 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.024 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.volume_copy_bps_limit = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.024 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.volume_dd_blocksize = 1M log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.024 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.025 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] backend_defaults.volumes_dir   = /var/lib/cinder/volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.025 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_concurrency.disable_process_locking = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.026 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_concurrency.lock_path     = /var/lib/cinder/tmp log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.026 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] barbican.auth_endpoint         = http://localhost/identity/v3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.027 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] barbican.barbican_api_version  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.027 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] barbican.barbican_endpoint     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.027 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] barbican.number_of_retries     = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.028 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] barbican.retry_delay           = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.028 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] barbican.verify_ssl            = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.028 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.admin_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.029 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.admin_tenant_name = admin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.029 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.admin_token = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.029 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.admin_user  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.030 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.auth_admin_prefix =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.030 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.auth_host   = 127.0.0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.031 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.auth_port   = 35357 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.031 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.auth_protocol = http log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.031 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.auth_section = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.032 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.auth_type   = password log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.032 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.auth_uri    = http://172.17.1.12:5000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.032 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.auth_version = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.033 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.cache       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.033 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.cafile      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.033 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.certfile    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.034 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.check_revocations_for_cached = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.034 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.delay_auth_decision = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.034 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.enforce_token_bind = permissive log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.035 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.hash_algorithms = ['md5'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.035 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.http_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.036 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.http_request_max_retries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.036 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.identity_uri = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.036 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.include_service_catalog = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.037 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.insecure    = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.037 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.keyfile     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.037 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.memcache_pool_conn_get_timeout = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.038 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.memcache_pool_dead_retry = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.038 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.memcache_pool_maxsize = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.038 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.memcache_pool_socket_timeout = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.039 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.memcache_pool_unused_timeout = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.039 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.memcache_secret_key = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.039 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.memcache_security_strategy = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.040 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.memcache_use_advanced_pool = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.040 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.memcached_servers = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.040 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.region_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.041 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.revocation_cache_time = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.041 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.service_token_roles = ['service'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.041 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.service_token_roles_required = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.042 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.signing_dir = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.042 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] keystone_authtoken.token_cache_time = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.042 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_versionedobjects.fatal_exception_format_errors = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.043 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_policy.policy_default_rule = default log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.043 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_policy.policy_dirs        = ['policy.d'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.044 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] oslo_policy.policy_file        = /etc/cinder/policy.json log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.044 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] key_manager.api_class          = cinder.keymgr.conf_key_mgr.ConfKeyManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.044 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] key_manager.fixed_key          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-24 13:00:25.045 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2889
>2018-06-24 13:00:25.076 139328 DEBUG cinder.volume.driver [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Initialized capabilities list: {'vendor_name': 'Open Source', 'multiattach': False, 'provisioned_capacity_gb': 0.0, 'volume_backend_name': 'tripleo_ceph', 'thin_provisioning_support': True, 'free_capacity_gb': 39.45, 'driver_version': '1.2.0', 'total_capacity_gb': 39.45, 'reserved_percentage': 0, 'max_over_subscription_ratio': 20.0, 'replication_enabled': False, 'properties': {'replication_enabled': {'type': 'boolean', 'description': u'Enables replication.', 'title': 'Replication'}, 'qos': {'type': 'boolean', 'description': u'Enables QoS.', 'title': 'QoS'}, 'compression': {'type': 'boolean', 'description': u'Enables compression.', 'title': 'Compression'}, 'thin_provisioning': {'type': 'boolean', 'description': u'Sets thin provisioning.', 'title': 'Thin Provisioning'}}, 'storage_protocol': 'ceph'}. init_capabilities /usr/lib/python2.7/site-packages/cinder/volume/driver.py:747
>2018-06-24 13:00:25.134 139328 DEBUG cinder.db.sqlalchemy.api [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Building query based on filter _process_snaps_filters /usr/lib/python2.7/site-packages/cinder/db/sqlalchemy/api.py:3006
>2018-06-24 13:00:25.147 139328 DEBUG cinder.image.image_utils [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Configuration image_conversion_dir is None or the path doesn't exist. cleanup_temporary_file /usr/lib/python2.7/site-packages/cinder/image/image_utils.py:568
>2018-06-24 13:00:25.148 139328 DEBUG cinder.volume.drivers.rbd [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:00:25.187 139328 DEBUG cinder.volume.drivers.rbd [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:00:25.226 139328 DEBUG cinder.manager [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:00:25.228 139328 WARNING oslo_config.cfg [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-24 13:00:25.229 139328 WARNING oslo_config.cfg [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-24 13:00:25.229 139328 WARNING oslo_config.cfg [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-24 13:00:25.261 139328 INFO cinder.volume.manager [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Driver initialization completed successfully.
>2018-06-24 13:00:25.271 139328 INFO cinder.manager [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Initiating service 5 cleanup
>2018-06-24 13:00:25.276 139328 INFO cinder.manager [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Service 5 cleanup completed.
>2018-06-24 13:00:25.277 139328 DEBUG cinder.service [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Creating RPC server for service cinder-volume start /usr/lib/python2.7/site-packages/cinder/service.py:219
>2018-06-24 13:00:25.282 139328 DEBUG cinder.service [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Pinning object versions for RPC server serializer to 1.27 start /usr/lib/python2.7/site-packages/cinder/service.py:226
>2018-06-24 13:00:25.498 139328 INFO cinder.volume.manager [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Initializing RPC dependent components of volume driver RBDDriver (1.2.0)
>2018-06-24 13:00:25.499 139328 DEBUG cinder.volume.drivers.rbd [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:00:25.537 139328 DEBUG cinder.volume.drivers.rbd [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:00:25.596 139328 INFO cinder.volume.manager [req-5412b44b-7b69-4daa-a560-9093d875a059 - - - - -] Driver post RPC initialization completed successfully.
>2018-06-24 13:01:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:01:39.599 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:01:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:01:39.607 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:01:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:02:39.599 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:02:39.600 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:02:39.608 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:02:39.609 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:02:39.648 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:03:39.601 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:03:39.603 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:03:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:03:39.614 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:03:39.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:04:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:04:39.598 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:04:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:04:39.608 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:04:39.652 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:05:39.596 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:05:39.597 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:05:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:05:39.608 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:05:39.650 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:06:39.592 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:06:39.595 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:06:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:06:39.605 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:06:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:07:39.588 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:07:39.590 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:07:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:07:39.598 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:07:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:08:39.587 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:08:39.588 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:08:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:08:39.597 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:08:39.638 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:09:39.588 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:09:39.590 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:09:39.598 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:09:39.599 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:09:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:10:21.284 139328 DEBUG cinder.volume.flows.manager.create_volume [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Volume reschedule parameters: True retry: {u'num_attempts': 1, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1075
>2018-06-24 13:10:21.347 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Flow 'volume_create_manager' (7377aa6d-9e34-4d8a-8634-910179fc5de8) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-24 13:10:21.350 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (95c4e9ea-fb48-474f-a774-d1f29e4be657) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:10:21.372 139328 DEBUG oslo_db.sqlalchemy.engines [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-24 13:10:21.395 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (95c4e9ea-fb48-474f-a774-d1f29e4be657) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:10:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1039738307',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=f2100b53-5f96-4023-8394-7bb280bd65a6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='dafc0bcea22b4e7c9db951f20c0a3ed1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:10:21Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:10:21Z,user_id='4d2a2ed08fdd46599f3797d764614457',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:10:21.397 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (04080082-ea10-45dd-8d10-c497a7b34d3b) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:10:21.399 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (04080082-ea10-45dd-8d10-c497a7b34d3b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:10:21.401 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (70993cd4-19a9-450c-b6b2-07d0cf1d327a) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:10:21.403 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (70993cd4-19a9-450c-b6b2-07d0cf1d327a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f2100b53-5f96-4023-8394-7bb280bd65a6', 'type': 'raw', 'volume_id': 'f2100b53-5f96-4023-8394-7bb280bd65a6'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:10:21.404 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f1c9f1b3-a942-4780-bd09-5327e29103e5) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:10:21.471 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f1c9f1b3-a942-4780-bd09-5327e29103e5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:10:21.473 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (275e1d93-0099-4dbb-bfd8-130b3172fee7) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:10:21.474 139328 INFO cinder.volume.flows.manager.create_volume [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Volume f2100b53-5f96-4023-8394-7bb280bd65a6: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-f2100b53-5f96-4023-8394-7bb280bd65a6'}
>2018-06-24 13:10:21.474 139328 DEBUG cinder.volume.drivers.rbd [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] creating volume 'volume-f2100b53-5f96-4023-8394-7bb280bd65a6' create_volume /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:655
>2018-06-24 13:10:21.475 139328 DEBUG cinder.volume.drivers.rbd [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:10:22.899 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (275e1d93-0099-4dbb-bfd8-130b3172fee7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:10:22.901 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ef4d966e-cba4-4e68-8d49-25954152bb1c) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:10:22.930 139328 INFO cinder.volume.flows.manager.create_volume [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Volume volume-f2100b53-5f96-4023-8394-7bb280bd65a6 (f2100b53-5f96-4023-8394-7bb280bd65a6): created successfully
>2018-06-24 13:10:22.933 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ef4d966e-cba4-4e68-8d49-25954152bb1c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:10:22.937 139328 DEBUG cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Flow 'volume_create_manager' (7377aa6d-9e34-4d8a-8634-910179fc5de8) transitioned into state 'SUCCESS' from state 'RUNNING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-24 13:10:22.937 139328 INFO cinder.volume.manager [req-4fc68c5e-e7af-4e91-9068-5930bdf34b81 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Created volume successfully.
>2018-06-24 13:10:35.049 139328 DEBUG oslo_concurrency.processutils [req-34cde1d7-0b06-4589-a680-079d66dd0e1f 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Running cmd (subprocess): ceph mon dump --format=json --id openstack --cluster ceph --conf /etc/ceph/ceph.conf execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:367
>2018-06-24 13:10:35.405 139328 DEBUG oslo_concurrency.processutils [req-34cde1d7-0b06-4589-a680-079d66dd0e1f 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] CMD "ceph mon dump --format=json --id openstack --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.357s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:404
>2018-06-24 13:10:35.407 139328 DEBUG cinder.volume.drivers.rbd [req-34cde1d7-0b06-4589-a680-079d66dd0e1f 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] connection data: {'driver_volume_type': 'rbd', 'data': {'secret_uuid': None, 'volume_id': 'f2100b53-5f96-4023-8394-7bb280bd65a6', 'auth_username': 'openstack', 'secret_type': 'ceph', 'name': 'volumes/volume-f2100b53-5f96-4023-8394-7bb280bd65a6', 'discard': True, 'keyring': None, 'cluster_name': 'ceph', 'hosts': [u'172.17.3.18', u'172.17.3.22', u'172.17.3.25'], 'auth_enabled': True, 'ports': [u'6789', u'6789', u'6789']}} initialize_connection /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:1136
>2018-06-24 13:10:35.407 139328 INFO cinder.volume.manager [req-34cde1d7-0b06-4589-a680-079d66dd0e1f 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Initialize volume connection completed successfully.
>2018-06-24 13:10:36.844 139328 DEBUG cinder.coordination [req-7042880e-9c0b-4219-aa16-8efc2f82c5d3 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Lock "/var/lib/cinder/cinder-f2100b53-5f96-4023-8394-7bb280bd65a6" acquired by "attach_volume" :: waited 0.001s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:175
>2018-06-24 13:10:36.945 139328 INFO cinder.volume.manager [req-7042880e-9c0b-4219-aa16-8efc2f82c5d3 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Attaching volume f2100b53-5f96-4023-8394-7bb280bd65a6 to instance 5bd5cba4-7580-49bf-ab56-d38c18405bf5 at mountpoint /dev/vdb on host None.
>2018-06-24 13:10:37.036 139328 INFO cinder.volume.manager [req-7042880e-9c0b-4219-aa16-8efc2f82c5d3 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Attach volume completed successfully.
>2018-06-24 13:10:37.037 139328 DEBUG cinder.coordination [req-7042880e-9c0b-4219-aa16-8efc2f82c5d3 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Lock "/var/lib/cinder/cinder-f2100b53-5f96-4023-8394-7bb280bd65a6" released by "attach_volume" :: held 0.193s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:187
>2018-06-24 13:10:39.590 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:10:39.591 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:10:39.598 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:10:39.598 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:10:39.636 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:10:39.672 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:11:18.783 139328 INFO cinder.volume.manager [req-2e00dbf7-bfce-40fe-aa17-fed611fb2e3d 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Terminate volume connection completed successfully.
>2018-06-24 13:11:20.055 139328 DEBUG cinder.coordination [req-d34e9f77-cb3d-4472-bb38-1a80e420b011 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Lock "/var/lib/cinder/cinder-f2100b53-5f96-4023-8394-7bb280bd65a6-detach_volume" acquired by "detach_volume" :: waited 0.001s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:175
>2018-06-24 13:11:20.102 139328 INFO cinder.volume.manager [req-d34e9f77-cb3d-4472-bb38-1a80e420b011 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Detaching volume f2100b53-5f96-4023-8394-7bb280bd65a6 from instance 5bd5cba4-7580-49bf-ab56-d38c18405bf5.
>2018-06-24 13:11:20.232 139328 INFO cinder.volume.manager [req-d34e9f77-cb3d-4472-bb38-1a80e420b011 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Detach volume completed successfully.
>2018-06-24 13:11:20.233 139328 DEBUG cinder.coordination [req-d34e9f77-cb3d-4472-bb38-1a80e420b011 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Lock "/var/lib/cinder/cinder-f2100b53-5f96-4023-8394-7bb280bd65a6-detach_volume" released by "detach_volume" :: held 0.178s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:187
>2018-06-24 13:11:20.630 139328 DEBUG cinder.coordination [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Lock "/var/lib/cinder/cinder-f2100b53-5f96-4023-8394-7bb280bd65a6-delete_volume" acquired by "delete_volume" :: waited 0.000s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:175
>2018-06-24 13:11:20.907 139328 DEBUG cinder.volume.drivers.rbd [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:11:20.963 139328 DEBUG cinder.volume.drivers.rbd [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] volume has no backup snaps _delete_backup_snaps /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:736
>2018-06-24 13:11:20.964 139328 DEBUG cinder.volume.drivers.rbd [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Volume volume-f2100b53-5f96-4023-8394-7bb280bd65a6 is not a clone. _get_clone_info /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:759
>2018-06-24 13:11:20.967 139328 DEBUG cinder.volume.drivers.rbd [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] deleting rbd volume volume-f2100b53-5f96-4023-8394-7bb280bd65a6 delete_volume /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:857
>2018-06-24 13:11:21.048 139328 DEBUG cinder.quota [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Created reservations ['6a7cb364-91cb-4ed9-a2f0-bd8c53ce1954', '28a6572f-4422-474f-ae58-6bf3360a519d'] reserve /usr/lib/python2.7/site-packages/cinder/quota.py:1029
>2018-06-24 13:11:21.082 139328 DEBUG cinder.volume.utils [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Can not find volume f2100b53-5f96-4023-8394-7bb280bd65a6 at notify usage _usage_from_volume /usr/lib/python2.7/site-packages/cinder/volume/utils.py:99
>2018-06-24 13:11:21.104 139328 DEBUG cinder.volume.drivers.rbd [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:11:21.147 139328 DEBUG cinder.volume.drivers.rbd [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:11:21.192 139328 DEBUG cinder.manager [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:11:21.201 139328 INFO cinder.volume.manager [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Deleted volume successfully.
>2018-06-24 13:11:21.207 139328 DEBUG cinder.coordination [req-1e485584-875b-4194-b7be-b1115757ee2a 4d2a2ed08fdd46599f3797d764614457 dafc0bcea22b4e7c9db951f20c0a3ed1 - default default] Lock "/var/lib/cinder/cinder-f2100b53-5f96-4023-8394-7bb280bd65a6-delete_volume" released by "delete_volume" :: held 0.577s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:187
>2018-06-24 13:11:39.595 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:11:39.596 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:12:39.594 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:12:39.595 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:12:39.602 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:12:39.603 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:12:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:13:39.593 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:13:39.594 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:13:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:13:39.607 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:13:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:14:39.595 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:14:39.597 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:14:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:14:39.606 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:14:39.648 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:15:39.599 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:15:39.600 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:15:39.638 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:16:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:16:39.598 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:16:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:16:39.607 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:16:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:17:39.595 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:17:39.596 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:17:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:17:39.607 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:17:39.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:18:39.595 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:18:39.596 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:18:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:18:39.604 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:18:39.643 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:19:39.594 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:19:39.595 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:19:39.603 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:19:39.604 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:19:39.646 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:20:39.596 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:20:39.597 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:20:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:20:39.605 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:20:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:21:39.593 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:21:39.595 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:21:39.605 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:21:39.606 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:21:39.646 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:22:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:22:39.599 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:22:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:22:39.606 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:22:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:23:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:23:39.599 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:23:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:23:39.610 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:23:39.666 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:24:39.599 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:24:39.601 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:24:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:24:39.610 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:24:39.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:25:39.603 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:25:39.604 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:25:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:25:39.612 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:25:39.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:26:39.600 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:26:39.601 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:26:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:26:39.609 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:26:39.654 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:27:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:27:39.599 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:27:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:27:39.607 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:27:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:28:39.594 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:28:39.595 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:28:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:28:39.604 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:28:39.643 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:29:39.593 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:29:39.595 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:29:39.602 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:29:39.603 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:29:39.657 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:30:39.594 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:30:39.595 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:30:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:30:39.605 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:30:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:31:39.594 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:31:39.596 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:31:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:31:39.605 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:31:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:32:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:32:39.599 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:32:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:32:39.607 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:32:39.648 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:33:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:33:39.598 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:33:39.605 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:33:39.605 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:33:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:34:39.595 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:34:39.596 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:34:39.603 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:34:39.604 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:34:39.654 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:35:39.594 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:35:39.596 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:35:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:35:39.605 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:35:39.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:36:39.594 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:36:39.596 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:36:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:36:39.605 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:36:39.646 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:37:39.596 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:37:39.598 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:37:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:37:39.613 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:37:39.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:38:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:38:39.599 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:38:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:38:39.608 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:38:39.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:39:39.600 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:39:39.602 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:39:39.615 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:39:39.616 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:39:39.666 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:40:39.602 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:40:39.604 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:40:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:40:39.615 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:40:39.664 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:41:39.602 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:41:39.604 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:41:39.611 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:41:39.612 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:41:39.668 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:42:39.603 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:42:39.605 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:42:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:42:39.613 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:42:39.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:43:39.602 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:43:39.604 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:43:39.610 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:43:39.611 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:43:39.654 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:44:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:44:39.609 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:44:39.618 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:44:39.619 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:44:39.668 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:45:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:45:39.609 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:45:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:45:39.617 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:45:39.656 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:45:39.656 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:45:39.656 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:45:39.656 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:45:39.656 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:45:39.656 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:45:39.656 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:45:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:45:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:45:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:45:49.663 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:45:49.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:45:49.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:45:49.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:45:49.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:45:49.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:45:49.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:45:49.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:45:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:45:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:45:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:46:09.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:46:09.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:46:09.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:46:09.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:46:09.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:46:09.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:46:09.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:46:09.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:46:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:46:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:46:09.745 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:46:39.603 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:46:39.605 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:46:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:46:39.614 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:46:39.656 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:46:39.656 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:46:39.656 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:46:39.656 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:46:39.656 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:46:39.656 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:46:39.656 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:46:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:46:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:46:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:46:49.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:46:49.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:46:49.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:46:49.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:46:49.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:46:49.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:46:49.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:46:49.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:46:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:46:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:46:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:47:09.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:47:09.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:47:09.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:47:09.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:47:09.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:47:09.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:47:09.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:47:09.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:47:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:47:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:47:09.744 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:47:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:47:39.608 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:47:39.620 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:47:39.621 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:47:39.658 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:47:39.658 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:47:39.658 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:47:39.658 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:47:39.658 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:47:39.658 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:47:39.658 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:47:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:47:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:47:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:47:49.662 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:47:49.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:47:49.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:47:49.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:47:49.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:47:49.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:47:49.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:47:49.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:47:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:47:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:47:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:48:09.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:48:09.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:48:09.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:48:09.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:48:09.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:48:09.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:48:09.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:48:09.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:48:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:48:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:48:09.753 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:48:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:48:39.606 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:48:39.615 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:48:39.616 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:48:39.654 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:48:39.654 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:48:39.654 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:48:39.654 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:48:39.654 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:48:39.654 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:48:39.654 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:48:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:48:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:48:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:48:49.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:48:49.694 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:48:49.694 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:48:49.694 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:48:49.694 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:48:49.694 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:48:49.694 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:48:49.694 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:48:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:48:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:48:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:49:09.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:49:09.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:49:09.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:49:09.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:49:09.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:49:09.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:49:09.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:49:09.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:49:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:49:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:49:09.741 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:49:39.603 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:49:39.605 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:49:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:49:39.615 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:49:39.655 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:49:39.655 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:49:39.655 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:49:39.655 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:49:39.655 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:49:39.655 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:49:39.655 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:49:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:49:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:49:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:49:49.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:49:49.691 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:49:49.691 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:49:49.691 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:49:49.691 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:49:49.691 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:49:49.691 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:49:49.691 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:49:49.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:49:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:49:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:50:09.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:50:09.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:50:09.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:50:09.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:50:09.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:50:09.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:50:09.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:50:09.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:50:09.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:50:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:50:09.735 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:50:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:50:39.608 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:50:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:50:39.617 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:50:39.650 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:50:39.650 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:50:39.650 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:50:39.650 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:50:39.650 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:50:39.650 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:50:39.650 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:50:39.653 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:50:39.653 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:50:39.653 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:50:49.654 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:50:49.694 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:50:49.694 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:50:49.694 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:50:49.694 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:50:49.694 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:50:49.694 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:50:49.694 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:50:49.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:50:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:50:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:51:09.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:51:09.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:51:09.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:51:09.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:51:09.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:51:09.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:51:09.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:51:09.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:51:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:51:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:51:09.738 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:51:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:51:39.611 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:51:39.621 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:51:39.622 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:51:39.658 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:51:39.658 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:51:39.658 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:51:39.658 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:51:39.658 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:51:39.658 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:51:39.658 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:51:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:51:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:51:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:51:49.663 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:51:49.702 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:51:49.702 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:51:49.702 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:51:49.702 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:51:49.702 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:51:49.702 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:51:49.702 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:51:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:51:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:51:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:52:09.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:52:09.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:52:09.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:52:09.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:52:09.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:52:09.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:52:09.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:52:09.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:52:09.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:52:09.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:52:09.747 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:52:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:52:39.611 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:52:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:52:39.619 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:52:39.661 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:52:39.661 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:52:39.661 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:52:39.661 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:52:39.661 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:52:39.661 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:52:39.661 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:52:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:52:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:52:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:52:49.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:52:49.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:52:49.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:52:49.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:52:49.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:52:49.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:52:49.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:52:49.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:52:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:52:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:52:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:53:09.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:53:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:53:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:53:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:53:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:53:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:53:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:53:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:53:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:53:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.216 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:53:09.836 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:53:39.608 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:53:39.610 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:53:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:53:39.618 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:53:39.655 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:53:39.655 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:53:39.655 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:53:39.655 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:53:39.655 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:53:39.655 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:53:39.655 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:53:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:53:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:53:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:53:49.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:53:49.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:53:49.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:53:49.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:53:49.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:53:49.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:53:49.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:53:49.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:53:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:53:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.098 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:53:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:54:09.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:54:09.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:54:09.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:54:09.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:54:09.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:54:09.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:54:09.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:54:09.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:54:09.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:54:09.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.142 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:54:09.761 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:54:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:54:39.613 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:54:39.626 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:54:39.627 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:54:39.667 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:54:39.667 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:54:39.667 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:54:39.667 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:54:39.667 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:54:39.667 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:54:39.667 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:54:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:54:39.670 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:54:39.670 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:54:49.671 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:54:49.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:54:49.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:54:49.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:54:49.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:54:49.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:54:49.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:54:49.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:54:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:54:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:54:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:55:09.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:55:09.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:55:09.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:55:09.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:55:09.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:55:09.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:55:09.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:55:09.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:55:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:55:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.128 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:55:09.755 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:55:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:55:39.619 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:55:39.626 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:55:39.627 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:55:39.665 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:55:39.665 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:55:39.665 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:55:39.665 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:55:39.665 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:55:39.665 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:55:39.665 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:55:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:55:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:55:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:55:49.668 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:55:49.702 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:55:49.702 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:55:49.702 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:55:49.702 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:55:49.702 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:55:49.702 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:55:49.702 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:55:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:55:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:55:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:56:09.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:56:09.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:56:09.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:56:09.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:56:09.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:56:09.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:56:09.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:56:09.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:56:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:56:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:56:09.751 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:56:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:56:39.614 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:56:39.621 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:56:39.622 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:56:39.661 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:56:39.661 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:56:39.661 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:56:39.661 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:56:39.661 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:56:39.661 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:56:39.661 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:56:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:56:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:56:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:56:49.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:56:49.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:56:49.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:56:49.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:56:49.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:56:49.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:56:49.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:56:49.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:56:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:56:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:56:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:57:09.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:57:09.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:57:09.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:57:09.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:57:09.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:57:09.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:57:09.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:57:09.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:57:09.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:57:09.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:57:09.742 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:57:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:57:39.614 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:57:39.621 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:57:39.621 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:57:39.660 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:57:39.660 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:57:39.660 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:57:39.660 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:57:39.660 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:57:39.660 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:57:39.660 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:57:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:57:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:57:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:57:49.663 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:57:49.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:57:49.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:57:49.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:57:49.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:57:49.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:57:49.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:57:49.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:57:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:57:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:57:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:58:09.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:58:09.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:58:09.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:58:09.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:58:09.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:58:09.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:58:09.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:58:09.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:58:09.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:58:09.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:58:09.741 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:58:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:58:39.615 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:58:39.624 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:58:39.624 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:58:39.668 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:58:39.668 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:58:39.668 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:58:39.668 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:58:39.668 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:58:39.668 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:58:39.668 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:58:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:58:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.047 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:58:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:58:49.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:58:49.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:58:49.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:58:49.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:58:49.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:58:49.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:58:49.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:58:49.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:58:49.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:58:49.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:58:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:59:08.339 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume reschedule parameters: True retry: {u'num_attempts': 1, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1075
>2018-06-24 13:59:08.358 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Flow 'volume_create_manager' (147bc965-d641-428a-b91a-8d234430a694) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-24 13:59:08.361 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (96a893cc-7f92-4fe5-977c-439b6350a172) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:08.382 139328 DEBUG oslo_db.sqlalchemy.engines [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-24 13:59:08.408 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (96a893cc-7f92-4fe5-977c-439b6350a172) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:08Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:08Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:59:08.411 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8ad5fcce-9c37-43d1-8ae9-6922efda7b9b) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:08.413 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8ad5fcce-9c37-43d1-8ae9-6922efda7b9b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:59:08.415 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a2504172-c94f-4088-a643-e6e821879898) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:08.417 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a2504172-c94f-4088-a643-e6e821879898) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b29f1542-c089-463c-8354-fa3d8de9030e', 'type': 'raw', 'volume_id': 'b29f1542-c089-463c-8354-fa3d8de9030e'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:59:08.419 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (da86c43e-1200-40e5-8081-89a4bba18273) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:08.446 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (da86c43e-1200-40e5-8081-89a4bba18273) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:59:08.449 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (86e72693-8c49-4932-83f5-cafa96616c8a) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:08.449 139328 INFO cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume b29f1542-c089-463c-8354-fa3d8de9030e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b29f1542-c089-463c-8354-fa3d8de9030e'}
>2018-06-24 13:59:08.450 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] creating volume 'volume-b29f1542-c089-463c-8354-fa3d8de9030e' create_volume /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:655
>2018-06-24 13:59:08.451 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:59:08.485 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:08.485 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:59:08.485 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:59:08.485 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:59:08.485 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:59:08.485 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:08.485 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:59:08.488 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:59:08.489 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:59:08.490 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:59:09.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:59:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:59:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:59:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:59:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:59:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:59:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:59:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.145 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:59:09.770 139328 ERROR oslo_service.periodic_task 
>2018-06-24 13:59:18.490 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:59:18.525 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:18.525 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:59:18.525 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:59:18.525 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:59:18.525 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:59:18.525 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:18.525 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:59:18.527 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:59:18.527 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:59:18.527 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:59:38.528 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:59:38.561 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:38.561 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:59:38.561 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:59:38.561 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:59:38.561 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:59:38.561 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:38.561 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:59:38.564 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:59:38.564 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:59:38.573 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (86e72693-8c49-4932-83f5-cafa96616c8a) transitioned into state 'FAILURE' from state 'RUNNING'
>5 predecessors (most recent first):
>  Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:08Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:08Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796f9451d0>}, 'provides': None}
>  |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:08Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:08Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(b29f1542-c089-463c-8354-fa3d8de9030e),volume_id=b29f1542-c089-463c-8354-fa3d8de9030e,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7f796f9451d0>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b29f1542-c089-463c-8354-fa3d8de9030e', 'type': 'raw', 'volume_id': 'b29f1542-c089-463c-8354-fa3d8de9030e'}}
>     |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:08Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:08Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'availability_zone': u'nova', u'reservations': [u'0e00f6b6-7527-4b5a-9184-b8ac9ffaa4d3', u'b4bce244-a3c7-4874-9e57-3e423c8bc6c6'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'3ee2014cc6684c8484b6a20dcabefd29', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestMinimumBasicScenario-volume-1056713959'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'availability_zone': u'nova', u'reservations': [u'0e00f6b6-7527-4b5a-9184-b8ac9ffaa4d3', u'b4bce244-a3c7-4874-9e57-3e423c8bc6c6'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'3ee2014cc6684c8484b6a20dcabefd29', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestMinimumBasicScenario-volume-1056713959'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'b29f1542-c089-463c-8354-fa3d8de9030e', u'size': 1, u'display_name': u'tempest-TestMinimumBasicScenario-volume-1056713959', u'display_description': None, u'cluster_name': None, u'name_id': u'b29f1542-c089-463c-8354-fa3d8de9030e', u'volume_admin_metadata': [], u'project_id': u'3ee2014cc6684c8484b6a20dcabefd29', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-b29f1542-c089-463c-8354-fa3d8de9030e', u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'bootable': False, u'created_at': u'2018-06-24T13:59:08.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'b29f1542-c089-463c-8354-fa3d8de9030e', u'group_id': None}, u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7f796f9451d0>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(b29f1542-c089-463c-8354-fa3d8de9030e),volume_id=b29f1542-c089-463c-8354-fa3d8de9030e,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None}
>        |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:08Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:08Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796f9451d0>}, 'provides': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:08Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:08Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)}
>           |__Flow 'volume_create_manager': VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     result = task.execute(**arguments)
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     ret = self.driver.create_volume(volume)
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     with RADOSClient(self) as client:
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     return _do_conn(pool, remote, timeout)
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     return r.call(f, *args, **kwargs)
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     raise attempt.get()
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 13:59:38.573 139328 ERROR cinder.volume.manager 
>2018-06-24 13:59:38.578 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (86e72693-8c49-4932-83f5-cafa96616c8a) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:38.580 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (86e72693-8c49-4932-83f5-cafa96616c8a) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 13:59:38.582 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (da86c43e-1200-40e5-8081-89a4bba18273) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:38.583 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (da86c43e-1200-40e5-8081-89a4bba18273) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 13:59:38.585 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a2504172-c94f-4088-a643-e6e821879898) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:38.587 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a2504172-c94f-4088-a643-e6e821879898) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 13:59:38.589 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8ad5fcce-9c37-43d1-8ae9-6922efda7b9b) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:38.590 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Updating volume b29f1542-c089-463c-8354-fa3d8de9030e with {'host': None, 'scheduled_at': datetime.datetime(2018, 6, 24, 13, 59, 38, 589964)}. _pre_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:121
>2018-06-24 13:59:38.597 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume b29f1542-c089-463c-8354-fa3d8de9030e: re-scheduling SchedulerAPI.create_volume attempt 1 due to Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster. _reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:148
>2018-06-24 13:59:38.606 139328 INFO cinder.rpc [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Automatically selected cinder-volume objects version 1.27 as minimum service version.
>2018-06-24 13:59:38.626 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume b29f1542-c089-463c-8354-fa3d8de9030e: re-scheduled _post_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:160
>2018-06-24 13:59:38.627 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:59:38.681 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:38.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:59:38.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:59:38.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:59:38.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:59:38.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:38.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:59:38.684 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:59:38.685 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 0.057 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:59:38.685 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:59:38.777 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume reschedule parameters: True retry: {u'num_attempts': 2, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1075
>2018-06-24 13:59:38.791 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Flow 'volume_create_manager' (ca46af6c-c063-40df-96fa-ea6fc3b8abd9) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-24 13:59:38.793 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0f9b35ea-0b95-4f85-8333-d134aa140ffb) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:38.809 139328 DEBUG oslo_db.sqlalchemy.engines [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-24 13:59:38.834 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0f9b35ea-0b95-4f85-8333-d134aa140ffb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:38Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:38Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:59:38.837 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7450468c-8a0d-4afe-a1cf-dca196c293fe) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:38.839 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7450468c-8a0d-4afe-a1cf-dca196c293fe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:59:38.841 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4db6ed02-e765-41f1-b15e-78e73759e18d) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:38.842 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4db6ed02-e765-41f1-b15e-78e73759e18d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b29f1542-c089-463c-8354-fa3d8de9030e', 'type': 'raw', 'volume_id': 'b29f1542-c089-463c-8354-fa3d8de9030e'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:59:38.844 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b6e7db03-c573-46a1-bd67-3ff86330fdeb) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:38.865 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b6e7db03-c573-46a1-bd67-3ff86330fdeb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 13:59:38.867 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a75d306b-fa03-4c9e-9c5d-1b7876c7b82c) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 13:59:38.868 139328 INFO cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume b29f1542-c089-463c-8354-fa3d8de9030e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b29f1542-c089-463c-8354-fa3d8de9030e'}
>2018-06-24 13:59:38.868 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] creating volume 'volume-b29f1542-c089-463c-8354-fa3d8de9030e' create_volume /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:655
>2018-06-24 13:59:38.869 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:59:38.902 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:38.902 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:59:38.902 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:59:38.902 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:59:38.902 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:59:38.902 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:38.902 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:59:38.904 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:59:38.904 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:59:38.905 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:59:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:59:39.612 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 13:59:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 13:59:39.620 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:59:39.658 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:39.658 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:59:39.658 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:59:39.658 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:59:39.658 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:59:39.658 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:39.658 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:59:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:59:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:59:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:59:48.686 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:59:48.721 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:48.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:59:48.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:59:48.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:59:48.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:59:48.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:48.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:59:48.724 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:59:48.724 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 10.096 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:59:48.724 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:59:48.905 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:59:48.938 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:48.938 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:59:48.938 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:59:48.938 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:59:48.938 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:59:48.938 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:48.938 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:59:48.940 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:59:48.941 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:59:48.941 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 13:59:49.662 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 13:59:49.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:49.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 13:59:49.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 13:59:49.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 13:59:49.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 13:59:49.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 13:59:49.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 13:59:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 13:59:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 13:59:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:00:08.725 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:08.762 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:08.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:08.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:08.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:08.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:08.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:08.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:08.765 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:08.765 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 30.137 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:08.767 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8ad5fcce-9c37-43d1-8ae9-6922efda7b9b) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
>2018-06-24 14:00:08.769 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (96a893cc-7f92-4fe5-977c-439b6350a172) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:08.771 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (96a893cc-7f92-4fe5-977c-439b6350a172) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 14:00:08.774 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Flow 'volume_create_manager' (147bc965-d641-428a-b91a-8d234430a694) transitioned into state 'REVERTED' from state 'RUNNING'
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "<string>", line 2, in create_volume
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     _run_flow()
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     flow_engine.run()
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     failures[0].reraise()
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     six.reraise(*self._exc_info)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     ret = self.driver.create_volume(volume)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     with RADOSClient(self) as client:
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     return _do_conn(pool, remote, timeout)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     return r.call(f, *args, **kwargs)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     raise attempt.get()
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:00:08.775 139328 ERROR oslo_messaging.rpc.server 
>2018-06-24 14:00:08.941 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:08.975 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:08.975 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:08.975 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:08.975 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:08.975 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:08.975 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:08.975 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:08.977 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:08.977 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:08.983 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a75d306b-fa03-4c9e-9c5d-1b7876c7b82c) transitioned into state 'FAILURE' from state 'RUNNING'
>5 predecessors (most recent first):
>  Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:38Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:38Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796f823910>}, 'provides': None}
>  |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:38Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:38Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(b29f1542-c089-463c-8354-fa3d8de9030e),volume_id=b29f1542-c089-463c-8354-fa3d8de9030e,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7f796f823910>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b29f1542-c089-463c-8354-fa3d8de9030e', 'type': 'raw', 'volume_id': 'b29f1542-c089-463c-8354-fa3d8de9030e'}}
>     |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:38Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:38Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'availability_zone': u'nova', u'reservations': [u'0e00f6b6-7527-4b5a-9184-b8ac9ffaa4d3', u'b4bce244-a3c7-4874-9e57-3e423c8bc6c6'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'3ee2014cc6684c8484b6a20dcabefd29', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestMinimumBasicScenario-volume-1056713959'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'availability_zone': u'nova', u'reservations': [u'0e00f6b6-7527-4b5a-9184-b8ac9ffaa4d3', u'b4bce244-a3c7-4874-9e57-3e423c8bc6c6'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'3ee2014cc6684c8484b6a20dcabefd29', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestMinimumBasicScenario-volume-1056713959'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'b29f1542-c089-463c-8354-fa3d8de9030e', u'size': 1, u'display_name': u'tempest-TestMinimumBasicScenario-volume-1056713959', u'display_description': None, u'cluster_name': None, u'name_id': u'b29f1542-c089-463c-8354-fa3d8de9030e', u'volume_admin_metadata': [], u'project_id': u'3ee2014cc6684c8484b6a20dcabefd29', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'admin_metadata': {}, u'host': None, u'glance_metadata': {}, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-b29f1542-c089-463c-8354-fa3d8de9030e', u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'bootable': False, u'created_at': u'2018-06-24T13:59:08.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'b29f1542-c089-463c-8354-fa3d8de9030e', u'group_id': None}, u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 2, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7f796f823910>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(b29f1542-c089-463c-8354-fa3d8de9030e),volume_id=b29f1542-c089-463c-8354-fa3d8de9030e,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None}
>        |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:38Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:38Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796f823910>}, 'provides': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T13:59:38Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T13:59:38Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)}
>           |__Flow 'volume_create_manager': VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     result = task.execute(**arguments)
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     ret = self.driver.create_volume(volume)
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     with RADOSClient(self) as client:
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     return _do_conn(pool, remote, timeout)
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     return r.call(f, *args, **kwargs)
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     raise attempt.get()
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:00:08.983 139328 ERROR cinder.volume.manager 
>2018-06-24 14:00:08.987 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a75d306b-fa03-4c9e-9c5d-1b7876c7b82c) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:08.989 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a75d306b-fa03-4c9e-9c5d-1b7876c7b82c) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 14:00:08.991 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b6e7db03-c573-46a1-bd67-3ff86330fdeb) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:08.992 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b6e7db03-c573-46a1-bd67-3ff86330fdeb) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 14:00:08.994 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4db6ed02-e765-41f1-b15e-78e73759e18d) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:08.996 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4db6ed02-e765-41f1-b15e-78e73759e18d) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 14:00:08.997 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7450468c-8a0d-4afe-a1cf-dca196c293fe) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:08.998 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Updating volume b29f1542-c089-463c-8354-fa3d8de9030e with {'host': None, 'scheduled_at': datetime.datetime(2018, 6, 24, 14, 0, 8, 998781)}. _pre_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:121
>2018-06-24 14:00:09.005 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume b29f1542-c089-463c-8354-fa3d8de9030e: re-scheduling SchedulerAPI.create_volume attempt 2 due to Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster. _reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:148
>2018-06-24 14:00:09.022 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume b29f1542-c089-463c-8354-fa3d8de9030e: re-scheduled _post_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:160
>2018-06-24 14:00:09.023 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:09.057 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:09.057 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:09.057 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:09.057 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:09.057 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:09.057 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:09.057 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:09.059 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:09.059 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:09.060 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:00:09.145 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume reschedule parameters: True retry: {u'num_attempts': 3, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1075
>2018-06-24 14:00:09.159 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Flow 'volume_create_manager' (c8ddde73-604a-49e3-9603-5cbe9702092e) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-24 14:00:09.162 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0ed92503-ed6d-4b7a-91b2-6e05cf32e211) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:09.181 139328 DEBUG oslo_db.sqlalchemy.engines [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-24 14:00:09.202 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0ed92503-ed6d-4b7a-91b2-6e05cf32e211) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T14:00:09Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T14:00:09Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 14:00:09.205 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eba82e4e-4db7-4aa5-9fa9-d399c8b89c71) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:09.207 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eba82e4e-4db7-4aa5-9fa9-d399c8b89c71) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 14:00:09.209 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (934c10b0-c8a7-4820-8733-2f87ca3487a0) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:09.211 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (934c10b0-c8a7-4820-8733-2f87ca3487a0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b29f1542-c089-463c-8354-fa3d8de9030e', 'type': 'raw', 'volume_id': 'b29f1542-c089-463c-8354-fa3d8de9030e'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 14:00:09.213 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ba4a7b93-b535-4b8c-a0a5-ef3517cb79a0) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:09.241 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ba4a7b93-b535-4b8c-a0a5-ef3517cb79a0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-24 14:00:09.243 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (60c5f342-99e6-4ea2-9e9e-b547cf1c25c6) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:09.243 139328 INFO cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume b29f1542-c089-463c-8354-fa3d8de9030e: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b29f1542-c089-463c-8354-fa3d8de9030e'}
>2018-06-24 14:00:09.244 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] creating volume 'volume-b29f1542-c089-463c-8354-fa3d8de9030e' create_volume /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:655
>2018-06-24 14:00:09.244 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:09.280 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:09.280 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:09.280 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:09.280 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:09.280 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:09.280 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:09.280 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:09.282 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:09.282 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:09.283 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:00:09.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:09.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:09.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:09.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:09.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:09.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:09.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:09.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:00:09.740 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:00:19.060 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:19.098 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:19.098 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:19.098 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:19.098 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:19.098 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:19.098 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:19.098 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:19.100 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:19.100 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:19.101 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:00:19.283 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:19.318 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:19.318 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:19.318 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:19.318 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:19.318 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:19.318 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:19.318 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:19.321 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:19.321 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:19.321 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:00:39.101 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:39.137 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:39.137 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:39.137 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:39.137 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:39.137 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:39.137 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:39.137 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:39.139 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:39.140 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:39.141 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7450468c-8a0d-4afe-a1cf-dca196c293fe) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
>2018-06-24 14:00:39.143 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0f9b35ea-0b95-4f85-8333-d134aa140ffb) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:39.145 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0f9b35ea-0b95-4f85-8333-d134aa140ffb) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 14:00:39.147 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Flow 'volume_create_manager' (ca46af6c-c063-40df-96fa-ea6fc3b8abd9) transitioned into state 'REVERTED' from state 'RUNNING'
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "<string>", line 2, in create_volume
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     _run_flow()
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     flow_engine.run()
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     failures[0].reraise()
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     six.reraise(*self._exc_info)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     ret = self.driver.create_volume(volume)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     with RADOSClient(self) as client:
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     return _do_conn(pool, remote, timeout)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     return r.call(f, *args, **kwargs)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     raise attempt.get()
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:00:39.148 139328 ERROR oslo_messaging.rpc.server 
>2018-06-24 14:00:39.322 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:39.354 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:39.354 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:39.354 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:39.354 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:39.354 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:39.354 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:39.354 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:39.357 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:39.357 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:39.363 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (60c5f342-99e6-4ea2-9e9e-b547cf1c25c6) transitioned into state 'FAILURE' from state 'RUNNING'
>5 predecessors (most recent first):
>  Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T14:00:09Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T14:00:09Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796fafed10>}, 'provides': None}
>  |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T14:00:09Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T14:00:09Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(b29f1542-c089-463c-8354-fa3d8de9030e),volume_id=b29f1542-c089-463c-8354-fa3d8de9030e,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7f796fafed10>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-b29f1542-c089-463c-8354-fa3d8de9030e', 'type': 'raw', 'volume_id': 'b29f1542-c089-463c-8354-fa3d8de9030e'}}
>     |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T14:00:09Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T14:00:09Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'availability_zone': u'nova', u'reservations': [u'0e00f6b6-7527-4b5a-9184-b8ac9ffaa4d3', u'b4bce244-a3c7-4874-9e57-3e423c8bc6c6'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'3ee2014cc6684c8484b6a20dcabefd29', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestMinimumBasicScenario-volume-1056713959'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'availability_zone': u'nova', u'reservations': [u'0e00f6b6-7527-4b5a-9184-b8ac9ffaa4d3', u'b4bce244-a3c7-4874-9e57-3e423c8bc6c6'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'3ee2014cc6684c8484b6a20dcabefd29', u'consistencygroup_id': None, u'size': 1, u'display_name': u'tempest-TestMinimumBasicScenario-volume-1056713959'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'b29f1542-c089-463c-8354-fa3d8de9030e', u'size': 1, u'display_name': u'tempest-TestMinimumBasicScenario-volume-1056713959', u'display_description': None, u'cluster_name': None, u'name_id': u'b29f1542-c089-463c-8354-fa3d8de9030e', u'volume_admin_metadata': [], u'project_id': u'3ee2014cc6684c8484b6a20dcabefd29', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'admin_metadata': {}, u'host': None, u'glance_metadata': {}, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-b29f1542-c089-463c-8354-fa3d8de9030e', u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'bootable': False, u'created_at': u'2018-06-24T13:59:08.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'b29f1542-c089-463c-8354-fa3d8de9030e', u'group_id': None}, u'user_id': u'239cfa9cffab4bc89f7b738ebd9f59c8', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 3, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7f796fafed10>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(b29f1542-c089-463c-8354-fa3d8de9030e),volume_id=b29f1542-c089-463c-8354-fa3d8de9030e,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None}
>        |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T14:00:09Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T14:00:09Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796fafed10>}, 'provides': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-24T13:59:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1056713959',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=b29f1542-c089-463c-8354-fa3d8de9030e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3ee2014cc6684c8484b6a20dcabefd29',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-24T14:00:09Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-24T14:00:09Z,user_id='239cfa9cffab4bc89f7b738ebd9f59c8',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)}
>           |__Flow 'volume_create_manager': VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     result = task.execute(**arguments)
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     ret = self.driver.create_volume(volume)
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     with RADOSClient(self) as client:
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     return _do_conn(pool, remote, timeout)
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     return r.call(f, *args, **kwargs)
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     raise attempt.get()
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:00:39.363 139328 ERROR cinder.volume.manager 
>2018-06-24 14:00:39.367 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (60c5f342-99e6-4ea2-9e9e-b547cf1c25c6) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:39.369 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (60c5f342-99e6-4ea2-9e9e-b547cf1c25c6) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 14:00:39.371 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ba4a7b93-b535-4b8c-a0a5-ef3517cb79a0) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:39.373 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ba4a7b93-b535-4b8c-a0a5-ef3517cb79a0) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 14:00:39.375 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (934c10b0-c8a7-4820-8733-2f87ca3487a0) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:39.377 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (934c10b0-c8a7-4820-8733-2f87ca3487a0) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 14:00:39.379 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eba82e4e-4db7-4aa5-9fa9-d399c8b89c71) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:00:39.380 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Updating volume b29f1542-c089-463c-8354-fa3d8de9030e with {'host': None, 'scheduled_at': datetime.datetime(2018, 6, 24, 14, 0, 39, 380211)}. _pre_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:121
>2018-06-24 14:00:39.387 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume b29f1542-c089-463c-8354-fa3d8de9030e: re-scheduling SchedulerAPI.create_volume attempt 3 due to Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster. _reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:148
>2018-06-24 14:00:39.403 139328 DEBUG cinder.volume.flows.manager.create_volume [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Volume b29f1542-c089-463c-8354-fa3d8de9030e: re-scheduled _post_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:160
>2018-06-24 14:00:39.404 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:39.436 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:39.436 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:39.436 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:39.436 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:39.436 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:39.436 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:39.436 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:39.438 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:39.439 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:39.439 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:00:39.611 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:00:39.612 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:00:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:00:39.620 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:39.668 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:39.668 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:39.668 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:39.668 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:39.668 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:39.668 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:39.668 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:39.671 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:39.671 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.05 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:00:49.440 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:49.478 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:49.478 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:49.478 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:49.478 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:49.478 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:49.478 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:49.478 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:49.481 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:49.481 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:49.482 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:00:49.672 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:00:49.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:49.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:00:49.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:00:49.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:00:49.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:00:49.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:00:49.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:00:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:00:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.092 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:00:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:01:09.482 139328 DEBUG cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:01:09.522 139328 ERROR cinder.volume.drivers.rbd [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:01:09.522 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:01:09.522 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:01:09.522 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:01:09.522 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:01:09.522 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:01:09.522 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:01:09.524 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:01:09.525 139328 DEBUG cinder.utils [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:01:09.527 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eba82e4e-4db7-4aa5-9fa9-d399c8b89c71) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
>2018-06-24 14:01:09.528 139328 DEBUG cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0ed92503-ed6d-4b7a-91b2-6e05cf32e211) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-24 14:01:09.530 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0ed92503-ed6d-4b7a-91b2-6e05cf32e211) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-24 14:01:09.532 139328 WARNING cinder.volume.manager [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Flow 'volume_create_manager' (c8ddde73-604a-49e3-9603-5cbe9702092e) transitioned into state 'REVERTED' from state 'RUNNING'
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server [req-63f464eb-1c45-47c5-bb28-65961e13c809 239cfa9cffab4bc89f7b738ebd9f59c8 3ee2014cc6684c8484b6a20dcabefd29 - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "<string>", line 2, in create_volume
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     _run_flow()
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     flow_engine.run()
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     failures[0].reraise()
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     six.reraise(*self._exc_info)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     ret = self.driver.create_volume(volume)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     with RADOSClient(self) as client:
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     return _do_conn(pool, remote, timeout)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     return r.call(f, *args, **kwargs)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     raise attempt.get()
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:01:09.533 139328 ERROR oslo_messaging.rpc.server 
>2018-06-24 14:01:09.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:01:09.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:01:09.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:01:09.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:01:09.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:01:09.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:01:09.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:01:09.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:01:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:01:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:01:09.750 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:01:39.611 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:01:39.614 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:01:39.626 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:01:39.627 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:01:39.666 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:01:39.666 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:01:39.666 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:01:39.666 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:01:39.666 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:01:39.666 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:01:39.666 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:01:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:01:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:01:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:01:49.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:01:49.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:01:49.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:01:49.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:01:49.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:01:49.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:01:49.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:01:49.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:01:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:01:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:01:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:02:09.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:02:09.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:02:09.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:02:09.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:02:09.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:02:09.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:02:09.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:02:09.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:02:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:02:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:02:09.759 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:02:39.610 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:02:39.611 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:02:39.622 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:02:39.623 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:02:39.657 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:02:39.657 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:02:39.657 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:02:39.657 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:02:39.657 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:02:39.657 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:02:39.657 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:02:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:02:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:02:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:02:49.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:02:49.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:02:49.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:02:49.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:02:49.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:02:49.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:02:49.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:02:49.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:02:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:02:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:02:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:03:09.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:03:09.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:03:09.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:03:09.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:03:09.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:03:09.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:03:09.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:03:09.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:03:09.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:03:09.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:03:09.747 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:03:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:03:39.610 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:03:39.624 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:03:39.625 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:03:39.670 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:03:39.670 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:03:39.670 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:03:39.670 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:03:39.670 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:03:39.670 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:03:39.670 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:03:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:03:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.047 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:03:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:03:49.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:03:49.707 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:03:49.707 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:03:49.707 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:03:49.707 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:03:49.707 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:03:49.707 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:03:49.707 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:03:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:03:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:03:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:04:09.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:04:09.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:04:09.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:04:09.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:04:09.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:04:09.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:04:09.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:04:09.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:04:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:04:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:04:09.751 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:04:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:04:39.615 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:04:39.626 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:04:39.627 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:04:39.666 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:04:39.666 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:04:39.666 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:04:39.666 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:04:39.666 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:04:39.666 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:04:39.666 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:04:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:04:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:04:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:04:49.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:04:49.706 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:04:49.706 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:04:49.706 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:04:49.706 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:04:49.706 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:04:49.706 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:04:49.706 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:04:49.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:04:49.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:04:49.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:05:09.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:05:09.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:05:09.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:05:09.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:05:09.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:05:09.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:05:09.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:05:09.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:05:09.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:05:09.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:05:09.748 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:05:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:05:39.610 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:05:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:05:39.617 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:05:39.653 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:05:39.653 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:05:39.653 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:05:39.653 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:05:39.653 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:05:39.653 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:05:39.653 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:05:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:05:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:05:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:05:49.657 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:05:49.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:05:49.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:05:49.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:05:49.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:05:49.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:05:49.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:05:49.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:05:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:05:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:05:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:06:09.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:06:09.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:06:09.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:06:09.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:06:09.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:06:09.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:06:09.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:06:09.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:06:09.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:06:09.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:06:09.730 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:06:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:06:39.613 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:06:39.649 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:06:39.649 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:06:39.649 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:06:39.649 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:06:39.649 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:06:39.649 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:06:39.649 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:06:39.652 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:06:39.652 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:06:39.653 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:06:49.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:06:49.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:06:49.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:06:49.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:06:49.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:06:49.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:06:49.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:06:49.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:06:49.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:06:49.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:06:49.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:07:09.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:07:09.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:07:09.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:07:09.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:07:09.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:07:09.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:07:09.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:07:09.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:07:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:07:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:07:09.735 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:07:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:07:39.617 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:07:39.628 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:07:39.629 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:07:39.665 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:07:39.665 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:07:39.665 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:07:39.665 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:07:39.665 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:07:39.665 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:07:39.665 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:07:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:07:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:07:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:07:49.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:07:49.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:07:49.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:07:49.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:07:49.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:07:49.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:07:49.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:07:49.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:07:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:07:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:07:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:08:09.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:08:09.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:08:09.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:08:09.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:08:09.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:08:09.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:08:09.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:08:09.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:08:09.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:08:09.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:08:09.746 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:08:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:08:39.618 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:08:39.625 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:08:39.626 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:08:39.662 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:08:39.662 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:08:39.662 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:08:39.662 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:08:39.662 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:08:39.662 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:08:39.662 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:08:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:08:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:08:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:08:49.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:08:49.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:08:49.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:08:49.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:08:49.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:08:49.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:08:49.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:08:49.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:08:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:08:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:08:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:09:09.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:09:09.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:09:09.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:09:09.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:09:09.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:09:09.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:09:09.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:09:09.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:09:09.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:09:09.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:09:09.738 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:09:39.620 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:09:39.620 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:09:39.628 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:09:39.628 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:09:39.664 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:09:39.664 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:09:39.664 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:09:39.664 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:09:39.664 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:09:39.664 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:09:39.664 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:09:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:09:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:09:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:09:49.668 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:09:49.702 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:09:49.702 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:09:49.702 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:09:49.702 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:09:49.702 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:09:49.702 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:09:49.702 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:09:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:09:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:09:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:10:09.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:10:09.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:10:09.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:10:09.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:10:09.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:10:09.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:10:09.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:10:09.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:10:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:10:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:10:09.744 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:10:39.615 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:10:39.616 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:10:39.623 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:10:39.624 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:10:39.658 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:10:39.658 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:10:39.658 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:10:39.658 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:10:39.658 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:10:39.658 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:10:39.658 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:10:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:10:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:10:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:10:49.661 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:10:49.705 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:10:49.705 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:10:49.705 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:10:49.705 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:10:49.705 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:10:49.705 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:10:49.705 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:10:49.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:10:49.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:10:49.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:11:09.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:11:09.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:11:09.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:11:09.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:11:09.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:11:09.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:11:09.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:11:09.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:11:09.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:11:09.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:11:09.747 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:11:39.618 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:11:39.619 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:11:39.627 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:11:39.627 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:11:39.663 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:11:39.663 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:11:39.663 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:11:39.663 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:11:39.663 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:11:39.663 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:11:39.663 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:11:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:11:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:11:39.666 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:11:49.666 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:11:49.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:11:49.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:11:49.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:11:49.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:11:49.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:11:49.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:11:49.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:11:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:11:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:11:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:12:09.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:12:09.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:12:09.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:12:09.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:12:09.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:12:09.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:12:09.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:12:09.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:12:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:12:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:12:09.744 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:12:39.615 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:12:39.616 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:12:39.622 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:12:39.623 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:12:39.657 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:12:39.657 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:12:39.657 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:12:39.657 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:12:39.657 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:12:39.657 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:12:39.657 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:12:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:12:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:12:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:12:49.661 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:12:49.694 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:12:49.694 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:12:49.694 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:12:49.694 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:12:49.694 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:12:49.694 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:12:49.694 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:12:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:12:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:12:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:13:09.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:13:09.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:13:09.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:13:09.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:13:09.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:13:09.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:13:09.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:13:09.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:13:09.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:13:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:13:09.737 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:13:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:13:39.620 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:13:39.627 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:13:39.628 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:13:39.665 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:13:39.665 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:13:39.665 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:13:39.665 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:13:39.665 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:13:39.665 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:13:39.665 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:13:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:13:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:13:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:13:49.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:13:49.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:13:49.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:13:49.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:13:49.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:13:49.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:13:49.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:13:49.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:13:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:13:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:13:49.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:14:09.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:14:09.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:14:09.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:14:09.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:14:09.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:14:09.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:14:09.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:14:09.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:14:09.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:14:09.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:14:09.747 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:14:39.618 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:14:39.619 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:14:39.627 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:14:39.628 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:14:39.665 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:14:39.665 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:14:39.665 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:14:39.665 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:14:39.665 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:14:39.665 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:14:39.665 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:14:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:14:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:14:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:14:49.668 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:14:49.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:14:49.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:14:49.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:14:49.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:14:49.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:14:49.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:14:49.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:14:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:14:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:14:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:15:09.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:15:09.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:15:09.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:15:09.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:15:09.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:15:09.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:15:09.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:15:09.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:15:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:15:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:15:09.744 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:15:39.615 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:15:39.617 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:15:39.628 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:15:39.628 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:15:39.663 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:15:39.663 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:15:39.663 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:15:39.663 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:15:39.663 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:15:39.663 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:15:39.663 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:15:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:15:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:15:39.666 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:15:49.666 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:15:49.702 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:15:49.702 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:15:49.702 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:15:49.702 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:15:49.702 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:15:49.702 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:15:49.702 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:15:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:15:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:15:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:16:09.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:16:09.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:16:09.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:16:09.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:16:09.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:16:09.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:16:09.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:16:09.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:16:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:16:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:16:09.750 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:16:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:16:39.613 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:16:39.620 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:16:39.621 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:16:39.657 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:16:39.657 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:16:39.657 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:16:39.657 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:16:39.657 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:16:39.657 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:16:39.657 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:16:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:16:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:16:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:16:49.661 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:16:49.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:16:49.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:16:49.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:16:49.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:16:49.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:16:49.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:16:49.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:16:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:16:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:16:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:17:09.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:17:09.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:17:09.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:17:09.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:17:09.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:17:09.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:17:09.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:17:09.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:17:09.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:17:09.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.139 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:17:09.760 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:17:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:17:39.612 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:17:39.620 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:17:39.621 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:17:39.655 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:17:39.655 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:17:39.655 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:17:39.655 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:17:39.655 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:17:39.655 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:17:39.655 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:17:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:17:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:17:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:17:49.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:17:49.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:17:49.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:17:49.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:17:49.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:17:49.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:17:49.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:17:49.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:17:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:17:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:17:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:18:09.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:18:09.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:18:09.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:18:09.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:18:09.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:18:09.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:18:09.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:18:09.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:18:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:18:09.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:18:09.733 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:18:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:18:39.615 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:18:39.622 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:18:39.623 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:18:39.657 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:18:39.657 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:18:39.657 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:18:39.657 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:18:39.657 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:18:39.657 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:18:39.657 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:18:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:18:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:18:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:18:49.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:18:49.695 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:18:49.695 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:18:49.695 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:18:49.695 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:18:49.695 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:18:49.695 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:18:49.695 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:18:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:18:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:18:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:19:09.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:19:09.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:19:09.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:19:09.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:19:09.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:19:09.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:19:09.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:19:09.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:19:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:19:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:19:09.740 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:19:39.611 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:19:39.612 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:19:39.620 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:19:39.620 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:19:39.654 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:19:39.654 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:19:39.654 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:19:39.654 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:19:39.654 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:19:39.654 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:19:39.654 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:19:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:19:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:19:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:19:49.657 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:19:49.690 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:19:49.690 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:19:49.690 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:19:49.690 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:19:49.690 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:19:49.690 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:19:49.690 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:19:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:19:49.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:19:49.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:20:09.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:20:09.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:20:09.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:20:09.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:20:09.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:20:09.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:20:09.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:20:09.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:20:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:20:09.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:20:09.733 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:20:39.615 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:20:39.616 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:21:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:21:39.614 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:21:39.626 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:21:39.626 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:21:39.659 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:21:39.659 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:21:39.659 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:21:39.659 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:21:39.659 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:21:39.659 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:21:39.659 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:21:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:21:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:21:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:21:49.663 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:21:49.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:21:49.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:21:49.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:21:49.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:21:49.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:21:49.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:21:49.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:21:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:21:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:21:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:22:09.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:22:09.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:22:09.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:22:09.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:22:09.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:22:09.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:22:09.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:22:09.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:22:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:22:09.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:22:09.738 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:22:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:22:39.610 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:22:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:22:39.617 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:22:39.650 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:22:39.650 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:22:39.650 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:22:39.650 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:22:39.650 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:22:39.650 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:22:39.650 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:22:39.652 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:22:39.652 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:22:39.653 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:22:49.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:22:49.690 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:22:49.690 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:22:49.690 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:22:49.690 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:22:49.690 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:22:49.690 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:22:49.690 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:22:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:22:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:22:49.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:23:09.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:23:09.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:23:09.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:23:09.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:23:09.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:23:09.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:23:09.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:23:09.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:23:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:23:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:23:09.732 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:23:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:23:39.614 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:23:39.621 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:23:39.622 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:23:39.655 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:23:39.655 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:23:39.655 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:23:39.655 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:23:39.655 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:23:39.655 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:23:39.655 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:23:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:23:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:23:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:23:49.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:23:49.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:23:49.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:23:49.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:23:49.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:23:49.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:23:49.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:23:49.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:23:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:23:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:23:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:24:09.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:24:09.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:24:09.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:24:09.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:24:09.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:24:09.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:24:09.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:24:09.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:24:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:24:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:24:09.740 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:24:39.611 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:24:39.612 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:24:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:24:39.620 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:24:39.658 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:24:39.658 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:24:39.658 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:24:39.658 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:24:39.658 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:24:39.658 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:24:39.658 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:24:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:24:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:24:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:24:49.661 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:24:49.694 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:24:49.694 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:24:49.694 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:24:49.694 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:24:49.694 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:24:49.694 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:24:49.694 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:24:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:24:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:24:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:25:09.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:25:09.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:25:09.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:25:09.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:25:09.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:25:09.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:25:09.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:25:09.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:25:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:25:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:25:09.740 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:25:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:25:39.612 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:25:39.621 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:25:39.622 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:25:39.667 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:25:39.667 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:25:39.667 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:25:39.667 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:25:39.667 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:25:39.667 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:25:39.667 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:25:39.670 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:25:39.670 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.048 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:25:39.671 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:25:49.671 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:25:49.704 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:25:49.704 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:25:49.704 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:25:49.704 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:25:49.704 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:25:49.704 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:25:49.704 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:25:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:25:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:25:49.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:26:09.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:26:09.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:26:09.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:26:09.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:26:09.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:26:09.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:26:09.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:26:09.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:26:09.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:26:09.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:26:09.747 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:26:39.608 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:26:39.610 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:26:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:26:39.619 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:26:39.654 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:26:39.654 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:26:39.654 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:26:39.654 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:26:39.654 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:26:39.654 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:26:39.654 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:26:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:26:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:26:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:26:49.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:26:49.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:26:49.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:26:49.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:26:49.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:26:49.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:26:49.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:26:49.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:26:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:26:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:26:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:27:09.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:27:09.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:27:09.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:27:09.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:27:09.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:27:09.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:27:09.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:27:09.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:27:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:27:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:27:09.737 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:27:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:27:39.609 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:27:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:27:39.616 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:27:39.653 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:27:39.653 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:27:39.653 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:27:39.653 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:27:39.653 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:27:39.653 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:27:39.653 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:27:39.654 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:27:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:27:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:27:49.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:27:49.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:27:49.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:27:49.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:27:49.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:27:49.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:27:49.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:27:49.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:27:49.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:27:49.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:27:49.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:28:09.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:28:09.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:28:09.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:28:09.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:28:09.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:28:09.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:28:09.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:28:09.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:28:09.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:28:09.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:28:09.731 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:28:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:28:39.613 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:28:39.620 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:28:39.621 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:28:39.654 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:28:39.654 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:28:39.654 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:28:39.654 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:28:39.654 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:28:39.654 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:28:39.654 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:28:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:28:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:28:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:28:49.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:28:49.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:28:49.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:28:49.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:28:49.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:28:49.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:28:49.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:28:49.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:28:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:28:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:28:49.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:29:09.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:29:09.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:29:09.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:29:09.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:29:09.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:29:09.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:29:09.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:29:09.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:29:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:29:09.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:29:09.736 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:29:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:29:39.608 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:29:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:29:39.618 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:29:39.652 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:29:39.652 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:29:39.652 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:29:39.652 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:29:39.652 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:29:39.652 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:29:39.652 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:29:39.654 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:29:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:29:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:29:49.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:29:49.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:29:49.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:29:49.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:29:49.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:29:49.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:29:49.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:29:49.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:29:49.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:29:49.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:29:49.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:30:09.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:30:09.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:30:09.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:30:09.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:30:09.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:30:09.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:30:09.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:30:09.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:30:09.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:30:09.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:30:09.730 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:30:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:30:39.613 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:30:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:30:39.620 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:30:39.654 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:30:39.654 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:30:39.654 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:30:39.654 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:30:39.654 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:30:39.654 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:30:39.654 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:30:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:30:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:30:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:30:49.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:30:49.691 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:30:49.691 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:30:49.691 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:30:49.691 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:30:49.691 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:30:49.691 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:30:49.691 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:30:49.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:30:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:30:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:31:09.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:31:09.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:31:09.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:31:09.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:31:09.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:31:09.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:31:09.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:31:09.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:31:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:31:09.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:31:09.733 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:31:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:31:39.615 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:31:39.622 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:31:39.623 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:31:39.656 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:31:39.656 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:31:39.656 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:31:39.656 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:31:39.656 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:31:39.656 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:31:39.656 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:31:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:31:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:31:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:31:49.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:31:49.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:31:49.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:31:49.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:31:49.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:31:49.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:31:49.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:31:49.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:31:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:31:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:31:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:32:09.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:32:09.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:32:09.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:32:09.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:32:09.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:32:09.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:32:09.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:32:09.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:32:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:32:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:32:09.732 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:32:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:32:39.614 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:32:39.621 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:32:39.622 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:32:39.659 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:32:39.659 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:32:39.659 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:32:39.659 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:32:39.659 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:32:39.659 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:32:39.659 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:32:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:32:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:32:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:32:49.664 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:32:49.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:32:49.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:32:49.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:32:49.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:32:49.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:32:49.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:32:49.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:32:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:32:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:32:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:33:09.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:33:09.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:33:09.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:33:09.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:33:09.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:33:09.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:33:09.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:33:09.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:33:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:33:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:33:09.738 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:33:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:33:39.610 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:33:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:33:39.620 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:33:39.656 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:33:39.656 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:33:39.656 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:33:39.656 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:33:39.656 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:33:39.656 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:33:39.656 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:33:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:33:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:33:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:33:49.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:33:49.691 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:33:49.691 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:33:49.691 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:33:49.691 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:33:49.691 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:33:49.691 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:33:49.691 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:33:49.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:33:49.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:33:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:34:09.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:34:09.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:34:09.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:34:09.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:34:09.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:34:09.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:34:09.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:34:09.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:34:09.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:34:09.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:34:09.735 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:34:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:34:39.607 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:34:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:34:39.619 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:34:39.657 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:34:39.657 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:34:39.657 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:34:39.657 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:34:39.657 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:34:39.657 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:34:39.657 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:34:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:34:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:34:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:34:49.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:34:49.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:34:49.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:34:49.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:34:49.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:34:49.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:34:49.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:34:49.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:34:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:34:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:34:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:35:09.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:35:09.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:35:09.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:35:09.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:35:09.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:35:09.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:35:09.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:35:09.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:35:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:35:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:35:09.735 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:35:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:35:39.607 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:35:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:35:39.617 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:35:39.650 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:35:39.650 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:35:39.650 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:35:39.650 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:35:39.650 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:35:39.650 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:35:39.650 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:35:39.652 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:35:39.652 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:35:39.652 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:35:49.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:35:49.688 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:35:49.688 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:35:49.688 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:35:49.688 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:35:49.688 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:35:49.688 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:35:49.688 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:35:49.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:35:49.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:35:49.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:36:09.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:36:09.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:36:09.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:36:09.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:36:09.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:36:09.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:36:09.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:36:09.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:36:09.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:36:09.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:36:09.730 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:36:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:36:39.613 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:36:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:36:39.620 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:36:39.653 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:36:39.653 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:36:39.653 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:36:39.653 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:36:39.653 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:36:39.653 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:36:39.653 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:36:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:36:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:36:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:36:49.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:36:49.691 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:36:49.691 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:36:49.691 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:36:49.691 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:36:49.691 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:36:49.691 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:36:49.691 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:36:49.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:36:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:36:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:37:09.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:37:09.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:37:09.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:37:09.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:37:09.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:37:09.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:37:09.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:37:09.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:37:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:37:09.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:37:09.733 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:37:39.615 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:37:39.616 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:37:39.624 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:37:39.625 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:37:39.659 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:37:39.659 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:37:39.659 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:37:39.659 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:37:39.659 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:37:39.659 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:37:39.659 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:37:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:37:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:37:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:37:49.662 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:37:49.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:37:49.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:37:49.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:37:49.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:37:49.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:37:49.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:37:49.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:37:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:37:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:37:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:38:09.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:38:09.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:38:09.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:38:09.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:38:09.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:38:09.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:38:09.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:38:09.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:38:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:38:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:38:09.740 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:38:39.611 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:38:39.612 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:38:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:38:39.620 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:38:39.665 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:38:39.665 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:38:39.665 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:38:39.665 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:38:39.665 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:38:39.665 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:38:39.665 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:38:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:38:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.048 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:38:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:38:49.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:38:49.704 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:38:49.704 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:38:49.704 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:38:49.704 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:38:49.704 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:38:49.704 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:38:49.704 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:38:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:38:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:38:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:39:09.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:39:09.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:39:09.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:39:09.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:39:09.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:39:09.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:39:09.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:39:09.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:39:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:39:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.137 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:39:09.757 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:39:39.608 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:39:39.609 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:39:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:39:39.618 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:39:39.657 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:39:39.657 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:39:39.657 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:39:39.657 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:39:39.657 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:39:39.657 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:39:39.657 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:39:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:39:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:39:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:39:49.661 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:39:49.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:39:49.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:39:49.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:39:49.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:39:49.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:39:49.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:39:49.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:39:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:39:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:39:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:40:09.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:40:09.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:40:09.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:40:09.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:40:09.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:40:09.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:40:09.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:40:09.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:40:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:40:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:40:09.743 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:40:39.605 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:40:39.605 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:40:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:40:39.613 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:40:39.653 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:40:39.653 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:40:39.653 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:40:39.653 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:40:39.653 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:40:39.653 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:40:39.653 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:40:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:40:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:40:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:40:49.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:40:49.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:40:49.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:40:49.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:40:49.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:40:49.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:40:49.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:40:49.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:40:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:40:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:40:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:41:09.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:41:09.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:41:09.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:41:09.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:41:09.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:41:09.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:41:09.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:41:09.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:41:09.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:41:09.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:41:09.732 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:41:39.603 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:41:39.604 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:41:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:41:39.612 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:41:39.659 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:41:39.659 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:41:39.659 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:41:39.659 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:41:39.659 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:41:39.659 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:41:39.659 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:41:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:41:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.049 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:41:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:41:49.664 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:41:49.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:41:49.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:41:49.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:41:49.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:41:49.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:41:49.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:41:49.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:41:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:41:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.087 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:41:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:42:09.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:42:09.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:42:09.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:42:09.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:42:09.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:42:09.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:42:09.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:42:09.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:42:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:42:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.127 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:42:09.741 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:42:39.602 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:42:39.602 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:42:39.610 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:42:39.610 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:42:39.646 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:42:39.646 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:42:39.646 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:42:39.646 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:42:39.646 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:42:39.646 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:42:39.646 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:42:39.649 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:42:39.649 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:42:39.650 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:42:49.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:42:49.686 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:42:49.686 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:42:49.686 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:42:49.686 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:42:49.686 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:42:49.686 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:42:49.686 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:42:49.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:42:49.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:42:49.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:43:09.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:43:09.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:43:09.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:43:09.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:43:09.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:43:09.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:43:09.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:43:09.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:43:09.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:43:09.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:43:09.727 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:43:39.599 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:43:39.599 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:43:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:43:39.607 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:43:39.639 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:43:39.639 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:43:39.639 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:43:39.639 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:43:39.639 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:43:39.639 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:43:39.639 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:43:39.640 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:43:39.641 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:43:39.641 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:43:49.641 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:43:49.673 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:43:49.673 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:43:49.673 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:43:49.673 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:43:49.673 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:43:49.673 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:43:49.673 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:43:49.675 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:43:49.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:43:49.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:44:09.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:44:09.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:44:09.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:44:09.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:44:09.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:44:09.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:44:09.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:44:09.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:44:09.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:44:09.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:44:09.719 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:44:39.601 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:44:39.601 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:44:39.608 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:44:39.609 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:44:39.642 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:44:39.642 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:44:39.642 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:44:39.642 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:44:39.642 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:44:39.642 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:44:39.642 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:44:39.644 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:44:39.644 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:44:39.644 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:44:49.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:44:49.681 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:44:49.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:44:49.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:44:49.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:44:49.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:44:49.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:44:49.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:44:49.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:44:49.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:44:49.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:45:09.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:45:09.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:45:09.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:45:09.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:45:09.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:45:09.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:45:09.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:45:09.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:45:09.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:45:09.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:45:09.724 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:45:39.605 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:45:39.606 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:45:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:45:39.614 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:45:39.646 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:45:39.646 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:45:39.646 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:45:39.646 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:45:39.646 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:45:39.646 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:45:39.646 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:45:39.648 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:45:39.649 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:45:39.649 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:45:49.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:45:49.682 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:45:49.682 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:45:49.682 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:45:49.682 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:45:49.682 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:45:49.682 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:45:49.682 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:45:49.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:45:49.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:45:49.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:46:09.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:46:09.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:46:09.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:46:09.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:46:09.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:46:09.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:46:09.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:46:09.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:46:09.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:46:09.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:46:09.723 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:46:39.605 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:46:39.606 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:46:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:46:39.619 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:46:39.654 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:46:39.654 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:46:39.654 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:46:39.654 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:46:39.654 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:46:39.654 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:46:39.654 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:46:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:46:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:46:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:46:49.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:46:49.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:46:49.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:46:49.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:46:49.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:46:49.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:46:49.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:46:49.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:46:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:46:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:46:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:47:09.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:47:09.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:47:09.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:47:09.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:47:09.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:47:09.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:47:09.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:47:09.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:47:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:47:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:47:09.740 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:47:39.601 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:47:39.602 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:47:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:47:39.609 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:47:39.642 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:47:39.642 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:47:39.642 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:47:39.642 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:47:39.642 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:47:39.642 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:47:39.642 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:47:39.645 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:47:39.646 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:47:39.646 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:47:49.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:47:49.686 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:47:49.686 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:47:49.686 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:47:49.686 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:47:49.686 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:47:49.686 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:47:49.686 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:47:49.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:47:49.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:47:49.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:48:09.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:48:09.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:48:09.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:48:09.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:48:09.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:48:09.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:48:09.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:48:09.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:48:09.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:48:09.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:48:09.727 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:48:39.599 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:48:39.599 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:48:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:48:39.606 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:48:39.640 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:48:39.640 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:48:39.640 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:48:39.640 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:48:39.640 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:48:39.640 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:48:39.640 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:48:39.642 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:48:39.642 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:48:39.642 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:48:49.643 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:48:49.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:48:49.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:48:49.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:48:49.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:48:49.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:48:49.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:48:49.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:48:49.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:48:49.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:48:49.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:49:09.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:49:09.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:49:09.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:49:09.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:49:09.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:49:09.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:49:09.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:49:09.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:49:09.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:49:09.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:49:09.719 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:49:39.600 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:49:39.601 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:49:39.608 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:49:39.608 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:49:39.642 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:49:39.642 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:49:39.642 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:49:39.642 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:49:39.642 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:49:39.642 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:49:39.642 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:49:39.644 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:49:39.645 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:49:39.645 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:49:49.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:49:49.684 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:49:49.684 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:49:49.684 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:49:49.684 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:49:49.684 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:49:49.684 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:49:49.684 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:49:49.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:49:49.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:49:49.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:50:09.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:50:09.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:50:09.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:50:09.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:50:09.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:50:09.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:50:09.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:50:09.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:50:09.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:50:09.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:50:09.725 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:50:39.596 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:50:39.597 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:50:39.608 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:50:39.609 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:50:39.647 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:50:39.647 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:50:39.647 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:50:39.647 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:50:39.647 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:50:39.647 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:50:39.647 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:50:39.649 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:50:39.650 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:50:39.650 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:50:49.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:50:49.683 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:50:49.683 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:50:49.683 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:50:49.683 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:50:49.683 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:50:49.683 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:50:49.683 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:50:49.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:50:49.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:50:49.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:51:09.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:51:09.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:51:09.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:51:09.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:51:09.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:51:09.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:51:09.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:51:09.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:51:09.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:51:09.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:51:09.722 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:51:39.594 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:51:39.594 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:51:39.601 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:51:39.602 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:51:39.635 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:51:39.635 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:51:39.635 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:51:39.635 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:51:39.635 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:51:39.635 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:51:39.635 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:51:39.637 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:51:39.638 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:51:39.638 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:51:49.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:51:49.674 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:51:49.674 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:51:49.674 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:51:49.674 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:51:49.674 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:51:49.674 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:51:49.674 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:51:49.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:51:49.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:51:49.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:52:09.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:52:09.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:52:09.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:52:09.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:52:09.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:52:09.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:52:09.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:52:09.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:52:09.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:52:09.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:52:09.715 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:52:39.596 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:52:39.597 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:52:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:52:39.604 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:52:39.640 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:52:39.640 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:52:39.640 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:52:39.640 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:52:39.640 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:52:39.640 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:52:39.640 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:52:39.642 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:52:39.643 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:52:39.643 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:52:49.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:52:49.677 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:52:49.677 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:52:49.677 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:52:49.677 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:52:49.677 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:52:49.677 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:52:49.677 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:52:49.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:52:49.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:52:49.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:53:09.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:53:09.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:53:09.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:53:09.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:53:09.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:53:09.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:53:09.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:53:09.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:53:09.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:53:09.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:53:09.717 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:53:39.598 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:53:39.598 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:53:39.605 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:53:39.606 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:53:39.641 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:53:39.641 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:53:39.641 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:53:39.641 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:53:39.641 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:53:39.641 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:53:39.641 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:53:39.643 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:53:39.644 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:53:39.644 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:53:49.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:53:49.677 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:53:49.677 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:53:49.677 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:53:49.677 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:53:49.677 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:53:49.677 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:53:49.677 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:53:49.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:53:49.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:53:49.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:54:09.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:54:09.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:54:09.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:54:09.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:54:09.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:54:09.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:54:09.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:54:09.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:54:09.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:54:09.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:54:09.720 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:54:39.602 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:54:39.603 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:54:39.610 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:54:39.610 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:54:39.644 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:54:39.644 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:54:39.644 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:54:39.644 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:54:39.644 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:54:39.644 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:54:39.644 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:54:39.647 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:54:39.647 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:54:39.648 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:54:49.648 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:54:49.681 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:54:49.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:54:49.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:54:49.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:54:49.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:54:49.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:54:49.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:54:49.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:54:49.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:54:49.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:55:09.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:55:09.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:55:09.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:55:09.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:55:09.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:55:09.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:55:09.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:55:09.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:55:09.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:55:09.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:55:09.725 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:55:39.597 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:55:39.598 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:55:39.605 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:55:39.606 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:55:39.640 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:55:39.640 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:55:39.640 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:55:39.640 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:55:39.640 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:55:39.640 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:55:39.640 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:55:39.643 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:55:39.643 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:55:39.644 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:55:49.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:55:49.677 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:55:49.677 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:55:49.677 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:55:49.677 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:55:49.677 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:55:49.677 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:55:49.677 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:55:49.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:55:49.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:55:49.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:56:09.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:56:09.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:56:09.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:56:09.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:56:09.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:56:09.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:56:09.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:56:09.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:56:09.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:56:09.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:56:09.718 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:56:39.600 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:56:39.601 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:56:39.608 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:56:39.609 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:56:39.644 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:56:39.644 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:56:39.644 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:56:39.644 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:56:39.644 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:56:39.644 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:56:39.644 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:56:39.646 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:56:39.646 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:56:39.646 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:56:49.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:56:49.680 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:56:49.680 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:56:49.680 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:56:49.680 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:56:49.680 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:56:49.680 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:56:49.680 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:56:49.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:56:49.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:56:49.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:57:09.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:57:09.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:57:09.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:57:09.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:57:09.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:57:09.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:57:09.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:57:09.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:57:09.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:57:09.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:57:09.721 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:57:39.603 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:57:39.604 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:57:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:57:39.615 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:57:39.648 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:57:39.648 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:57:39.648 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:57:39.648 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:57:39.648 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:57:39.648 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:57:39.648 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:57:39.650 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:57:39.650 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:57:39.651 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:57:49.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:57:49.684 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:57:49.684 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:57:49.684 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:57:49.684 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:57:49.684 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:57:49.684 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:57:49.684 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:57:49.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:57:49.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:57:49.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:58:09.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:58:09.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:58:09.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:58:09.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:58:09.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:58:09.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:58:09.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:58:09.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:58:09.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:58:09.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:58:09.725 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:58:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:58:39.607 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:58:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:58:39.615 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:58:39.652 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:58:39.652 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:58:39.652 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:58:39.652 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:58:39.652 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:58:39.652 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:58:39.652 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:58:39.654 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:58:39.654 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:58:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:58:49.655 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:58:49.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:58:49.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:58:49.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:58:49.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:58:49.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:58:49.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:58:49.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:58:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:58:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:58:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:59:09.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:59:09.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:59:09.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:59:09.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:59:09.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:59:09.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:59:09.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:59:09.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:59:09.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:59:09.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 14:59:09.734 139328 ERROR oslo_service.periodic_task 
>2018-06-24 14:59:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:59:39.607 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 14:59:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 14:59:39.615 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:59:39.651 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:59:39.651 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:59:39.651 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:59:39.651 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:59:39.651 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:59:39.651 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:59:39.651 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:59:39.654 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:59:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:59:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 14:59:49.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 14:59:49.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 14:59:49.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 14:59:49.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 14:59:49.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 14:59:49.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 14:59:49.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 14:59:49.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 14:59:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 14:59:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 14:59:49.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:00:09.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:00:09.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:00:09.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:00:09.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:00:09.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:00:09.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:00:09.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:00:09.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:00:09.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:00:09.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:00:09.733 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:00:39.604 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:00:39.605 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:00:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:00:39.614 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:00:39.652 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:00:39.652 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:00:39.652 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:00:39.652 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:00:39.652 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:00:39.652 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:00:39.652 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:00:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:00:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:00:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:00:49.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:00:49.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:00:49.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:00:49.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:00:49.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:00:49.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:00:49.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:00:49.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:00:49.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:00:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:00:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:01:09.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:01:09.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:01:09.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:01:09.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:01:09.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:01:09.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:01:09.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:01:09.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:01:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:01:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:01:09.744 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:01:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:01:39.608 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:01:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:01:39.614 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:01:39.650 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:01:39.650 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:01:39.650 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:01:39.650 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:01:39.650 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:01:39.650 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:01:39.650 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:01:39.652 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:01:39.652 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:01:39.653 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:01:49.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:01:49.686 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:01:49.686 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:01:49.686 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:01:49.686 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:01:49.686 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:01:49.686 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:01:49.686 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:01:49.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:01:49.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:01:49.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:02:09.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:02:09.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:02:09.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:02:09.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:02:09.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:02:09.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:02:09.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:02:09.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:02:09.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:02:09.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:02:09.726 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:02:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:02:39.608 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:02:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:02:39.616 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:02:39.651 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:02:39.651 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:02:39.651 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:02:39.651 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:02:39.651 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:02:39.651 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:02:39.651 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:02:39.653 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:02:39.653 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:02:39.654 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:02:49.654 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:02:49.690 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:02:49.690 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:02:49.690 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:02:49.690 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:02:49.690 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:02:49.690 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:02:49.690 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:02:49.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:02:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:02:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:03:09.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:03:09.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:03:09.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:03:09.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:03:09.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:03:09.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:03:09.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:03:09.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:03:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:03:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:03:09.735 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:03:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:03:39.608 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:03:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:03:39.616 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:03:39.658 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:03:39.658 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:03:39.658 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:03:39.658 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:03:39.658 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:03:39.658 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:03:39.658 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:03:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:03:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:03:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:03:49.662 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:03:49.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:03:49.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:03:49.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:03:49.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:03:49.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:03:49.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:03:49.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:03:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:03:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:03:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:04:09.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:04:09.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:04:09.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:04:09.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:04:09.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:04:09.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:04:09.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:04:09.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:04:09.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:04:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:04:09.739 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:04:39.611 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:04:39.612 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:04:39.627 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:04:39.628 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:04:39.683 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:04:39.683 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:04:39.683 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:04:39.683 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:04:39.683 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:04:39.683 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:04:39.683 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:04:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:04:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.061 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:04:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:04:49.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:04:49.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:04:49.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:04:49.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:04:49.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:04:49.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:04:49.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:04:49.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:04:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:04:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.098 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:04:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:05:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:05:09.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:05:09.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:05:09.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:05:09.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:05:09.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:05:09.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:05:09.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:05:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:05:09.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.136 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:05:09.764 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:05:39.606 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:05:39.607 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:05:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:05:39.617 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:05:39.658 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:05:39.658 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:05:39.658 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:05:39.658 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:05:39.658 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:05:39.658 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:05:39.658 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:05:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:05:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:05:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:05:49.661 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:05:49.695 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:05:49.695 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:05:49.695 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:05:49.695 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:05:49.695 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:05:49.695 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:05:49.695 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:05:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:05:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:05:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:06:09.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:06:09.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:06:09.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:06:09.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:06:09.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:06:09.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:06:09.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:06:09.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:06:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:06:09.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:06:09.736 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:06:39.607 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:06:39.608 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:06:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:06:39.616 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:06:39.650 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:06:39.650 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:06:39.650 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:06:39.650 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:06:39.650 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:06:39.650 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:06:39.650 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:06:39.653 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:06:39.653 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:06:39.654 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:06:49.655 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:06:49.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:06:49.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:06:49.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:06:49.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:06:49.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:06:49.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:06:49.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:06:49.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:06:49.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:06:49.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:07:09.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:07:09.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:07:09.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:07:09.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:07:09.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:07:09.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:07:09.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:07:09.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:07:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:07:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:07:09.740 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:07:39.611 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:07:39.612 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:07:39.618 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:07:39.619 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:07:39.653 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:07:39.653 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:07:39.653 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:07:39.653 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:07:39.653 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:07:39.653 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:07:39.653 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:07:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:07:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:07:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:07:49.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:07:49.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:07:49.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:07:49.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:07:49.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:07:49.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:07:49.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:07:49.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:07:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:07:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:07:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:08:09.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:08:09.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:08:09.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:08:09.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:08:09.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:08:09.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:08:09.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:08:09.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:08:09.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:08:09.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:08:09.735 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:08:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:08:39.617 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:08:39.632 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:08:39.633 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:08:39.667 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:08:39.667 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:08:39.667 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:08:39.667 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:08:39.667 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:08:39.667 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:08:39.667 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:08:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:08:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:08:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:08:49.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:08:49.704 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:08:49.704 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:08:49.704 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:08:49.704 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:08:49.704 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:08:49.704 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:08:49.704 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:08:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:08:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:08:49.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:09:09.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:09:09.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:09:09.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:09:09.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:09:09.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:09:09.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:09:09.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:09:09.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:09:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:09:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:09:09.745 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:09:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:09:39.617 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:09:39.624 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:09:39.625 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:09:39.657 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:09:39.657 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:09:39.657 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:09:39.657 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:09:39.657 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:09:39.657 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:09:39.657 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:09:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:09:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:09:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:09:49.661 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:09:49.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:09:49.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:09:49.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:09:49.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:09:49.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:09:49.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:09:49.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:09:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:09:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:09:49.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:10:09.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:10:09.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:10:09.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:10:09.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:10:09.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:10:09.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:10:09.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:10:09.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:10:09.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:10:09.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:10:09.734 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:10:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:10:39.616 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:10:39.622 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:10:39.623 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:10:39.657 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:10:39.657 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:10:39.657 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:10:39.657 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:10:39.657 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:10:39.657 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:10:39.657 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:10:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:10:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:10:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:10:49.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:10:49.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:10:49.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:10:49.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:10:49.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:10:49.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:10:49.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:10:49.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:10:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:10:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:10:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:11:09.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:11:09.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:11:09.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:11:09.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:11:09.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:11:09.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:11:09.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:11:09.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:11:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:11:09.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:11:09.733 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:11:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:11:39.615 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:11:39.623 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:11:39.624 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:11:39.656 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:11:39.656 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:11:39.656 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:11:39.656 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:11:39.656 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:11:39.656 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:11:39.656 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:11:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:11:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:11:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:11:49.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:11:49.694 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:11:49.694 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:11:49.694 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:11:49.694 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:11:49.694 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:11:49.694 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:11:49.694 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:11:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:11:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:11:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:12:09.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:12:09.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:12:09.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:12:09.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:12:09.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:12:09.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:12:09.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:12:09.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:12:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:12:09.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:12:09.733 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:12:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:12:39.615 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:12:39.656 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:12:39.656 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:12:39.656 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:12:39.656 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:12:39.656 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:12:39.656 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:12:39.656 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:12:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:12:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:12:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:12:49.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:12:49.695 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:12:49.695 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:12:49.695 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:12:49.695 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:12:49.695 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:12:49.695 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:12:49.695 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:12:49.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:12:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:12:49.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:13:09.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:13:09.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:13:09.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:13:09.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:13:09.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:13:09.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:13:09.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:13:09.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:13:09.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:13:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:13:09.737 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:13:39.618 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:13:39.619 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:13:39.627 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:13:39.627 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:13:39.664 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:13:39.664 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:13:39.664 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:13:39.664 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:13:39.664 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:13:39.664 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:13:39.664 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:13:39.666 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:13:39.666 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:13:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:13:49.667 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:13:49.701 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:13:49.701 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:13:49.701 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:13:49.701 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:13:49.701 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:13:49.701 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:13:49.701 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:13:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:13:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:13:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:14:09.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:14:09.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:14:09.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:14:09.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:14:09.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:14:09.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:14:09.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:14:09.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:14:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:14:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:14:09.739 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:14:39.621 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:14:39.621 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:14:39.629 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:14:39.629 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:14:39.681 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:14:39.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:14:39.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:14:39.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:14:39.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:14:39.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:14:39.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:14:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:14:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.054 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:14:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:14:49.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:14:49.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:14:49.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:14:49.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:14:49.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:14:49.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:14:49.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:14:49.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:14:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:14:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.09 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:14:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:15:09.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:15:09.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:15:09.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:15:09.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:15:09.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:15:09.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:15:09.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:15:09.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:15:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:15:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:15:09.756 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:15:39.618 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:15:39.619 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:16:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:16:39.618 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:16:39.625 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:16:39.625 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:16:39.662 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:16:39.662 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:16:39.662 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:16:39.662 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:16:39.662 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:16:39.662 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:16:39.662 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:16:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:16:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:16:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:16:49.666 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:16:49.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:16:49.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:16:49.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:16:49.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:16:49.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:16:49.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:16:49.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:16:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:16:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:16:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:17:09.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:17:09.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:17:09.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:17:09.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:17:09.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:17:09.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:17:09.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:17:09.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:17:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:17:09.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:17:09.736 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:17:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:17:39.618 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:17:39.629 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:17:39.629 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:17:39.665 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:17:39.665 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:17:39.665 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:17:39.665 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:17:39.665 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:17:39.665 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:17:39.665 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:17:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:17:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:17:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:17:49.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:17:49.702 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:17:49.702 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:17:49.702 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:17:49.702 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:17:49.702 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:17:49.702 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:17:49.702 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:17:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:17:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:17:49.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:18:09.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:18:09.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:18:09.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:18:09.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:18:09.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:18:09.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:18:09.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:18:09.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:18:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:18:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:18:09.743 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:18:39.615 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:18:39.616 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:18:39.623 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:18:39.624 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:18:39.663 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:18:39.663 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:18:39.663 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:18:39.663 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:18:39.663 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:18:39.663 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:18:39.663 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:18:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:18:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:18:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:18:49.666 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:18:49.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:18:49.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:18:49.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:18:49.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:18:49.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:18:49.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:18:49.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:18:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:18:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:18:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:19:09.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:19:09.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:19:09.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:19:09.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:19:09.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:19:09.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:19:09.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:19:09.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:19:09.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:19:09.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:19:09.743 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:19:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:19:39.615 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:19:39.623 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:19:39.623 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:19:39.658 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:19:39.658 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:19:39.658 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:19:39.658 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:19:39.658 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:19:39.658 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:19:39.658 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:19:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:19:39.660 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:19:39.661 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:19:49.661 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:19:49.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:19:49.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:19:49.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:19:49.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:19:49.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:19:49.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:19:49.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:19:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:19:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:19:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:20:09.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:20:09.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:20:09.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:20:09.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:20:09.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:20:09.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:20:09.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:20:09.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:20:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:20:09.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:20:09.738 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:20:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:20:39.610 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:20:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:20:39.619 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:20:39.652 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:20:39.652 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:20:39.652 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:20:39.652 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:20:39.652 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:20:39.652 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:20:39.652 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:20:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:20:39.655 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:20:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:20:49.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:20:49.690 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:20:49.690 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:20:49.690 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:20:49.690 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:20:49.690 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:20:49.690 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:20:49.690 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:20:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:20:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:20:49.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:21:09.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:21:09.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:21:09.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:21:09.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:21:09.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:21:09.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:21:09.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:21:09.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:21:09.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:21:09.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:21:09.731 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:21:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:21:39.613 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:21:39.620 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:21:39.621 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:21:39.660 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:21:39.660 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:21:39.660 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:21:39.660 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:21:39.660 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:21:39.660 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:21:39.660 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:21:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:21:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:21:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:21:49.664 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:21:49.704 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:21:49.704 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:21:49.704 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:21:49.704 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:21:49.704 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:21:49.704 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:21:49.704 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:21:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:21:49.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:21:49.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:22:09.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:22:09.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:22:09.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:22:09.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:22:09.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:22:09.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:22:09.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:22:09.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:22:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:22:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:22:09.752 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:22:39.613 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:22:39.615 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:22:39.621 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:22:39.621 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:22:39.654 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:22:39.654 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:22:39.654 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:22:39.654 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:22:39.654 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:22:39.654 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:22:39.654 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:22:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:22:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:22:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:22:49.657 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:22:49.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:22:49.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:22:49.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:22:49.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:22:49.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:22:49.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:22:49.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:22:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:22:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:22:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:23:09.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:23:09.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:23:09.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:23:09.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:23:09.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:23:09.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:23:09.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:23:09.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:23:09.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:23:09.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:23:09.736 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:23:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:23:39.618 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:23:39.626 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:23:39.627 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:23:39.661 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:23:39.661 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:23:39.661 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:23:39.661 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:23:39.661 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:23:39.661 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:23:39.661 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:23:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:23:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:23:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:23:49.664 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:23:49.700 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:23:49.700 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:23:49.700 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:23:49.700 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:23:49.700 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:23:49.700 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:23:49.700 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:23:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:23:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:23:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:24:09.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:24:09.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:24:09.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:24:09.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:24:09.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:24:09.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:24:09.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:24:09.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:24:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:24:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:24:09.741 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:24:39.622 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:24:39.623 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:24:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:24:39.632 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:24:39.668 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:24:39.668 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:24:39.668 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:24:39.668 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:24:39.668 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:24:39.668 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:24:39.668 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:24:39.670 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:24:39.670 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:24:39.671 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:24:49.671 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:24:49.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:24:49.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:24:49.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:24:49.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:24:49.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:24:49.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:24:49.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:24:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:24:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:24:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:25:09.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:25:09.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:25:09.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:25:09.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:25:09.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:25:09.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:25:09.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:25:09.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:25:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:25:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:25:09.755 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:25:39.626 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:25:39.627 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:25:39.633 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:25:39.634 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:25:39.667 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:25:39.667 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:25:39.667 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:25:39.667 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:25:39.667 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:25:39.667 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:25:39.667 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:25:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:25:39.670 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:25:39.670 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:25:49.671 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:25:49.706 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:25:49.706 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:25:49.706 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:25:49.706 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:25:49.706 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:25:49.706 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:25:49.706 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:25:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:25:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:25:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:26:09.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:26:09.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:26:09.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:26:09.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:26:09.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:26:09.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:26:09.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:26:09.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:26:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:26:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:26:09.752 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:26:39.623 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:26:39.624 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:26:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:26:39.631 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:26:39.664 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:26:39.664 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:26:39.664 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:26:39.664 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:26:39.664 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:26:39.664 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:26:39.664 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:26:39.666 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:26:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:26:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:26:49.667 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:26:49.707 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:26:49.707 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:26:49.707 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:26:49.707 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:26:49.707 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:26:49.707 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:26:49.707 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:26:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:26:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:26:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:27:09.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:27:09.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:27:09.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:27:09.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:27:09.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:27:09.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:27:09.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:27:09.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:27:09.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:27:09.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:27:09.748 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:27:39.619 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:27:39.621 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:27:39.629 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:27:39.629 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:27:39.664 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:27:39.664 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:27:39.664 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:27:39.664 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:27:39.664 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:27:39.664 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:27:39.664 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:27:39.666 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:27:39.666 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:27:39.666 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:27:49.667 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:27:49.701 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:27:49.701 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:27:49.701 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:27:49.701 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:27:49.701 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:27:49.701 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:27:49.701 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:27:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:27:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:27:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:28:09.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:28:09.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:28:09.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:28:09.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:28:09.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:28:09.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:28:09.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:28:09.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:28:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:28:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:28:09.744 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:28:39.615 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:28:39.616 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:28:39.626 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:28:39.627 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:28:39.662 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:28:39.662 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:28:39.662 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:28:39.662 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:28:39.662 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:28:39.662 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:28:39.662 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:28:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:28:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:28:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:28:49.666 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:28:49.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:28:49.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:28:49.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:28:49.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:28:49.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:28:49.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:28:49.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:28:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:28:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:28:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:29:09.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:29:09.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:29:09.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:29:09.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:29:09.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:29:09.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:29:09.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:29:09.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:29:09.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:29:09.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:29:09.739 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:29:39.620 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:29:39.621 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:29:39.628 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:29:39.628 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:29:39.661 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:29:39.661 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:29:39.661 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:29:39.661 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:29:39.661 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:29:39.661 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:29:39.661 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:29:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:29:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:29:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:29:49.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:29:49.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:29:49.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:29:49.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:29:49.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:29:49.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:29:49.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:29:49.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:29:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:29:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:29:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:30:09.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:30:09.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:30:09.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:30:09.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:30:09.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:30:09.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:30:09.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:30:09.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:30:09.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:30:09.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:30:09.739 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:30:39.620 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:30:39.621 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:30:39.628 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:30:39.628 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:30:39.665 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:30:39.665 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:30:39.665 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:30:39.665 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:30:39.665 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:30:39.665 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:30:39.665 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:30:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:30:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:30:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:30:49.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:30:49.701 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:30:49.701 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:30:49.701 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:30:49.701 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:30:49.701 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:30:49.701 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:30:49.701 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:30:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:30:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:30:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:31:09.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:31:09.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:31:09.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:31:09.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:31:09.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:31:09.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:31:09.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:31:09.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:31:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:31:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:31:09.744 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:31:39.616 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:31:39.617 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:31:39.624 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:31:39.625 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:31:39.661 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:31:39.661 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:31:39.661 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:31:39.661 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:31:39.661 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:31:39.661 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:31:39.661 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:31:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:31:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:31:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:31:49.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:31:49.700 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:31:49.700 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:31:49.700 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:31:49.700 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:31:49.700 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:31:49.700 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:31:49.700 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:31:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:31:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:31:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:32:09.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:32:09.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:32:09.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:32:09.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:32:09.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:32:09.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:32:09.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:32:09.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:32:09.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:32:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:32:09.739 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:32:39.612 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:32:39.612 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:32:39.620 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:32:39.621 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:32:39.660 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:32:39.660 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:32:39.660 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:32:39.660 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:32:39.660 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:32:39.660 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:32:39.660 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:32:39.662 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:32:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:32:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:32:49.664 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:32:49.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:32:49.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:32:49.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:32:49.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:32:49.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:32:49.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:32:49.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:32:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:32:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:32:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:33:09.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:33:09.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:33:09.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:33:09.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:33:09.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:33:09.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:33:09.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:33:09.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:33:09.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:33:09.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:33:09.736 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:33:39.608 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:33:39.609 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:33:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:33:39.617 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:33:39.654 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:33:39.654 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:33:39.654 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:33:39.654 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:33:39.654 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:33:39.654 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:33:39.654 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:33:39.656 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:33:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:33:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:33:49.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:33:49.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:33:49.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:33:49.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:33:49.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:33:49.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:33:49.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:33:49.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:33:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:33:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:33:49.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:34:09.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:34:09.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:34:09.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:34:09.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:34:09.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:34:09.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:34:09.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:34:09.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:34:09.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:34:09.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:34:09.748 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:34:39.609 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:34:39.610 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:34:39.617 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:34:39.618 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:34:39.655 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:34:39.655 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:34:39.655 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:34:39.655 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:34:39.655 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:34:39.655 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:34:39.655 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:34:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:34:39.658 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:34:39.659 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:34:49.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:34:49.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:34:49.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:34:49.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:34:49.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:34:49.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:34:49.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:34:49.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:34:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:34:49.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:34:49.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:35:09.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:35:09.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:35:09.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:35:09.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:35:09.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:35:09.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:35:09.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:35:09.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:35:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:35:09.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:35:09.732 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:35:39.614 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:35:39.615 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:35:39.622 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:35:39.622 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:35:39.655 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:35:39.655 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:35:39.655 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:35:39.655 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:35:39.655 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:35:39.655 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:35:39.655 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:35:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:35:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:35:39.657 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:35:49.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:35:49.695 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:35:49.695 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:35:49.695 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:35:49.695 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:35:49.695 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:35:49.695 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:35:49.695 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:35:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:35:49.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:35:49.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:36:09.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:36:09.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:36:09.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:36:09.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:36:09.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:36:09.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:36:09.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:36:09.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:36:09.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:36:09.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:36:09.737 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:36:39.618 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:36:39.619 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:36:39.626 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:36:39.626 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:36:39.660 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:36:39.660 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:36:39.660 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:36:39.660 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:36:39.660 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:36:39.660 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:36:39.660 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:36:39.663 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:36:39.664 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:36:39.665 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:36:49.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:36:49.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:36:49.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:36:49.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:36:49.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:36:49.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:36:49.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:36:49.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:36:49.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:36:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:36:49.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:37:09.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:37:09.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:37:09.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:37:09.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:37:09.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:37:09.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:37:09.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:37:09.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:37:09.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:37:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:37:09.740 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:37:39.621 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:37:39.622 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:37:39.630 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:37:39.630 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:37:39.666 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:37:39.666 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:37:39.666 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:37:39.666 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:37:39.666 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:37:39.666 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:37:39.666 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:37:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:37:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:37:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:37:49.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:37:49.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:37:49.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:37:49.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:37:49.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:37:49.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:37:49.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:37:49.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:37:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:37:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.096 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:37:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:38:09.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:38:09.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:38:09.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:38:09.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:38:09.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:38:09.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:38:09.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:38:09.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:38:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:38:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.132 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:38:09.763 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:38:39.624 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:38:39.626 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:38:39.632 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:38:39.633 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:38:39.665 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:38:39.665 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:38:39.665 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:38:39.665 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:38:39.665 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:38:39.665 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:38:39.665 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:38:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:38:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:38:39.669 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:38:49.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:38:49.705 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:38:49.705 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:38:49.705 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:38:49.705 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:38:49.705 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:38:49.705 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:38:49.705 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:38:49.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:38:49.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:38:49.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:39:09.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:39:09.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:39:09.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:39:09.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:39:09.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:39:09.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:39:09.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:39:09.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:39:09.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:39:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:39:09.744 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:39:39.625 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:39:39.626 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:39:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:39:39.635 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:39:39.671 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:39:39.671 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:39:39.671 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:39:39.671 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:39:39.671 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:39:39.671 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:39:39.671 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:39:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:39:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:39:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:39:49.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:39:49.708 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:39:49.708 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:39:49.708 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:39:49.708 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:39:49.708 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:39:49.708 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:39:49.708 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:39:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:39:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:39:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:40:09.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:40:09.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:40:09.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:40:09.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:40:09.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:40:09.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:40:09.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:40:09.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:40:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:40:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:40:09.753 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:40:39.625 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:40:39.627 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:40:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:40:39.636 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:40:39.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:40:39.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:40:39.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:40:39.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:40:39.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:40:39.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:40:39.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:40:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:40:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:40:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:40:49.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:40:49.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:40:49.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:40:49.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:40:49.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:40:49.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:40:49.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:40:49.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:40:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:40:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:40:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:41:09.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:41:09.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:41:09.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:41:09.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:41:09.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:41:09.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:41:09.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:41:09.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:41:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:41:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:41:09.754 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:41:39.625 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:41:39.627 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:41:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:41:39.636 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:41:39.675 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:41:39.675 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:41:39.675 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:41:39.675 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:41:39.675 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:41:39.675 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:41:39.675 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:41:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:41:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:41:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:41:49.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:41:49.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:41:49.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:41:49.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:41:49.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:41:49.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:41:49.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:41:49.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:41:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:41:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:41:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:42:09.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:42:09.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:42:09.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:42:09.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:42:09.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:42:09.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:42:09.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:42:09.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:42:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:42:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:42:09.751 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:42:39.622 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:42:39.623 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:42:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:42:39.631 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:42:39.666 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:42:39.666 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:42:39.666 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:42:39.666 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:42:39.666 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:42:39.666 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:42:39.666 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:42:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:42:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:42:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:42:49.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:42:49.701 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:42:49.701 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:42:49.701 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:42:49.701 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:42:49.701 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:42:49.701 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:42:49.701 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:42:49.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:42:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:42:49.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:43:09.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:43:09.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:43:09.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:43:09.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:43:09.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:43:09.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:43:09.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:43:09.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:43:09.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:43:09.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:43:09.741 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:43:39.622 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:43:39.623 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:43:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:43:39.632 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:43:39.665 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:43:39.665 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:43:39.665 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:43:39.665 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:43:39.665 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:43:39.665 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:43:39.665 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:43:39.667 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:43:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:43:39.668 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:43:49.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:43:49.707 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:43:49.707 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:43:49.707 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:43:49.707 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:43:49.707 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:43:49.707 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:43:49.707 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:43:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:43:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:43:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:44:09.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:44:09.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:44:09.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:44:09.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:44:09.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:44:09.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:44:09.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:44:09.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:44:09.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:44:09.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:44:09.746 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:44:39.627 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:44:39.628 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:44:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:44:39.637 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:44:39.673 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:44:39.673 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:44:39.673 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:44:39.673 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:44:39.673 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:44:39.673 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:44:39.673 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:44:39.675 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:44:39.675 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:44:39.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:44:49.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:44:49.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:44:49.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:44:49.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:44:49.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:44:49.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:44:49.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:44:49.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:44:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:44:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:44:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:45:09.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:45:09.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:45:09.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:45:09.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:45:09.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:45:09.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:45:09.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:45:09.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:45:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:45:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:45:09.750 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:45:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:45:39.632 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:45:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:45:39.640 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:45:39.674 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:45:39.674 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:45:39.674 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:45:39.674 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:45:39.674 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:45:39.674 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:45:39.674 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:45:39.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:45:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:45:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:45:49.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:45:49.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:45:49.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:45:49.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:45:49.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:45:49.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:45:49.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:45:49.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:45:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:45:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:45:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:46:09.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:46:09.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:46:09.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:46:09.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:46:09.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:46:09.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:46:09.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:46:09.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:46:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:46:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:46:09.753 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:46:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:46:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:46:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:46:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:46:39.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:46:39.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:46:39.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:46:39.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:46:39.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:46:39.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:46:39.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:46:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:46:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.054 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:46:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:46:49.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:46:49.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:46:49.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:46:49.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:46:49.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:46:49.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:46:49.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:46:49.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:46:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:46:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.091 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:46:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:47:09.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:47:09.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:47:09.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:47:09.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:47:09.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:47:09.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:47:09.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:47:09.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:47:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:47:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:47:09.777 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:47:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:47:39.639 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:47:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:47:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:47:39.678 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:47:39.678 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:47:39.678 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:47:39.678 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:47:39.678 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:47:39.678 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:47:39.678 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:47:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:47:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:47:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:47:49.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:47:49.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:47:49.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:47:49.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:47:49.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:47:49.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:47:49.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:47:49.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:47:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:47:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.067 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:47:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:48:09.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:48:09.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:48:09.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:48:09.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:48:09.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:48:09.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:48:09.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:48:09.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:48:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:48:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:48:09.753 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:48:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:48:39.639 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:48:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:48:39.664 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:48:39.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:48:39.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:48:39.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:48:39.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:48:39.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:48:39.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:48:39.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:48:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:48:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:48:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:48:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:48:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:48:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:48:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:48:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:48:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:48:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:48:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:48:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:48:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:48:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:49:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:49:09.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:49:09.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:49:09.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:49:09.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:49:09.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:49:09.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:49:09.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:49:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:49:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.155 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:49:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:49:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:49:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:49:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:49:39.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:49:39.684 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:49:39.684 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:49:39.684 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:49:39.684 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:49:39.684 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:49:39.684 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:49:39.684 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:49:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:49:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:49:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:49:49.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:49:49.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:49:49.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:49:49.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:49:49.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:49:49.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:49:49.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:49:49.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:49:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:49:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:49:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:50:09.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:50:09.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:50:09.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:50:09.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:50:09.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:50:09.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:50:09.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:50:09.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:50:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:50:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:50:09.769 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:50:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:50:39.641 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:50:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:50:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:50:39.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:50:39.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:50:39.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:50:39.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:50:39.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:50:39.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:50:39.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:50:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:50:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:50:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:50:49.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:50:49.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:50:49.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:50:49.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:50:49.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:50:49.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:50:49.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:50:49.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:50:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:50:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.068 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:50:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:51:09.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:51:09.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:51:09.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:51:09.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:51:09.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:51:09.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:51:09.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:51:09.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:51:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:51:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:51:09.755 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:51:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:51:39.637 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:51:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:51:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:51:39.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:51:39.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:51:39.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:51:39.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:51:39.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:51:39.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:51:39.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:51:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:51:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:51:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:51:49.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:51:49.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:51:49.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:51:49.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:51:49.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:51:49.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:51:49.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:51:49.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:51:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:51:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:51:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:52:09.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:52:09.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:52:09.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:52:09.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:52:09.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:52:09.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:52:09.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:52:09.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:52:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:52:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:52:09.754 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:52:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:52:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:52:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:52:39.643 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:52:39.677 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:52:39.677 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:52:39.677 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:52:39.677 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:52:39.677 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:52:39.677 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:52:39.677 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:52:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:52:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:52:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:52:49.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:52:49.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:52:49.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:52:49.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:52:49.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:52:49.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:52:49.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:52:49.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:52:49.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:52:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:52:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:53:09.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:53:09.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:53:09.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:53:09.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:53:09.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:53:09.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:53:09.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:53:09.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:53:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:53:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:53:09.759 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:53:39.630 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:53:39.632 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:53:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:53:39.640 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:53:39.673 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:53:39.673 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:53:39.673 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:53:39.673 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:53:39.673 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:53:39.673 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:53:39.673 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:53:39.675 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:53:39.675 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:53:39.675 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:53:49.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:53:49.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:53:49.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:53:49.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:53:49.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:53:49.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:53:49.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:53:49.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:53:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:53:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:53:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:54:09.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:54:09.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:54:09.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:54:09.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:54:09.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:54:09.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:54:09.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:54:09.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:54:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:54:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:54:09.750 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:54:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:54:39.632 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:54:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:54:39.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:54:39.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:54:39.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:54:39.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:54:39.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:54:39.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:54:39.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:54:39.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:54:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:54:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:54:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:54:49.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:54:49.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:54:49.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:54:49.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:54:49.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:54:49.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:54:49.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:54:49.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:54:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:54:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:54:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:55:09.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:55:09.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:55:09.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:55:09.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:55:09.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:55:09.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:55:09.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:55:09.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:55:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:55:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:55:09.757 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:55:39.628 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:55:39.629 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:55:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:55:39.636 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:55:39.669 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:55:39.669 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:55:39.669 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:55:39.669 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:55:39.669 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:55:39.669 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:55:39.669 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:55:39.671 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:55:39.671 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:55:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:55:49.672 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:55:49.706 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:55:49.706 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:55:49.706 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:55:49.706 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:55:49.706 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:55:49.706 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:55:49.706 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:55:49.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:55:49.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:55:49.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:56:09.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:56:09.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:56:09.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:56:09.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:56:09.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:56:09.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:56:09.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:56:09.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:56:09.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:56:09.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:56:09.745 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:56:39.627 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:56:39.628 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:56:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:56:39.636 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:56:39.669 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:56:39.669 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:56:39.669 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:56:39.669 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:56:39.669 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:56:39.669 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:56:39.669 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:56:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:56:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:56:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:56:49.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:56:49.708 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:56:49.708 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:56:49.708 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:56:49.708 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:56:49.708 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:56:49.708 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:56:49.708 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:56:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:56:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:56:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:57:09.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:57:09.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:57:09.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:57:09.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:57:09.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:57:09.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:57:09.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:57:09.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:57:09.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:57:09.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:57:09.747 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:57:39.628 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:57:39.629 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:57:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:57:39.637 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:57:39.675 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:57:39.675 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:57:39.675 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:57:39.675 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:57:39.675 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:57:39.675 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:57:39.675 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:57:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:57:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:57:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:57:49.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:57:49.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:57:49.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:57:49.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:57:49.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:57:49.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:57:49.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:57:49.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:57:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:57:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:57:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:58:09.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:58:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:58:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:58:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:58:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:58:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:58:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:58:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:58:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:58:09.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.133 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:58:09.771 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:58:39.633 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:58:39.634 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:58:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:58:39.643 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:58:39.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:58:39.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:58:39.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:58:39.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:58:39.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:58:39.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:58:39.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:58:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:58:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:58:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:58:49.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:58:49.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:58:49.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:58:49.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:58:49.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:58:49.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:58:49.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:58:49.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:58:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:58:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:58:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:59:09.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:59:09.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:59:09.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:59:09.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:59:09.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:59:09.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:59:09.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:59:09.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:59:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:59:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 15:59:09.758 139328 ERROR oslo_service.periodic_task 
>2018-06-24 15:59:39.629 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:59:39.630 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 15:59:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 15:59:39.637 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:59:39.671 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:59:39.671 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:59:39.671 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:59:39.671 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:59:39.671 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:59:39.671 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:59:39.671 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:59:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:59:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:59:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 15:59:49.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 15:59:49.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 15:59:49.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 15:59:49.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 15:59:49.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 15:59:49.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 15:59:49.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 15:59:49.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 15:59:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 15:59:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 15:59:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:00:09.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:00:09.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:00:09.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:00:09.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:00:09.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:00:09.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:00:09.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:00:09.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:00:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:00:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:00:09.750 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:00:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:00:39.633 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:00:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:00:39.642 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:00:39.675 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:00:39.675 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:00:39.675 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:00:39.675 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:00:39.675 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:00:39.675 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:00:39.675 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:00:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:00:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:00:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:00:49.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:00:49.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:00:49.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:00:49.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:00:49.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:00:49.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:00:49.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:00:49.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:00:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:00:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:00:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:01:09.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:01:09.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:01:09.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:01:09.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:01:09.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:01:09.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:01:09.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:01:09.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:01:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:01:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:01:09.758 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:01:39.629 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:01:39.630 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:01:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:01:39.638 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:01:39.675 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:01:39.675 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:01:39.675 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:01:39.675 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:01:39.675 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:01:39.675 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:01:39.675 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:01:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:01:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:01:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:01:49.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:01:49.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:01:49.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:01:49.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:01:49.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:01:49.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:01:49.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:01:49.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:01:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:01:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:01:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:02:09.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:02:09.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:02:09.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:02:09.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:02:09.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:02:09.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:02:09.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:02:09.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:02:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:02:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:02:09.750 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:02:39.632 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:02:39.633 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:02:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:02:39.642 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:02:39.678 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:02:39.678 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:02:39.678 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:02:39.678 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:02:39.678 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:02:39.678 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:02:39.678 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:02:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:02:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:02:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:02:49.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:02:49.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:02:49.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:02:49.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:02:49.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:02:49.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:02:49.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:02:49.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:02:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:02:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:02:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:03:09.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:03:09.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:03:09.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:03:09.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:03:09.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:03:09.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:03:09.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:03:09.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:03:09.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:03:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:03:09.773 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:03:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:03:39.635 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:03:39.671 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:03:39.671 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:03:39.671 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:03:39.671 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:03:39.671 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:03:39.671 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:03:39.671 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:03:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:03:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:03:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:03:49.674 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:03:49.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:03:49.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:03:49.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:03:49.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:03:49.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:03:49.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:03:49.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:03:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:03:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:03:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:04:09.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:04:09.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:04:09.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:04:09.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:04:09.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:04:09.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:04:09.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:04:09.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:04:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:04:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:04:09.752 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:04:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:04:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:04:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:04:39.642 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:04:39.674 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:04:39.674 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:04:39.674 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:04:39.674 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:04:39.674 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:04:39.674 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:04:39.674 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:04:39.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:04:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:04:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:04:49.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:04:49.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:04:49.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:04:49.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:04:49.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:04:49.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:04:49.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:04:49.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:04:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:04:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:04:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:05:09.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:05:09.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:05:09.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:05:09.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:05:09.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:05:09.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:05:09.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:05:09.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:05:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:05:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:05:09.749 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:05:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:05:39.632 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:05:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:05:39.642 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:05:39.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:05:39.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:05:39.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:05:39.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:05:39.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:05:39.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:05:39.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:05:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:05:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.052 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:05:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:05:49.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:05:49.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:05:49.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:05:49.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:05:49.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:05:49.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:05:49.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:05:49.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:05:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:05:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.091 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:05:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:06:09.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:06:09.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:06:09.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:06:09.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:06:09.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:06:09.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:06:09.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:06:09.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:06:09.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:06:09.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:06:09.772 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:06:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:06:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:06:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:06:39.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:06:39.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:06:39.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:06:39.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:06:39.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:06:39.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:06:39.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:06:39.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:06:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:06:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:06:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:06:49.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:06:49.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:06:49.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:06:49.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:06:49.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:06:49.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:06:49.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:06:49.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:06:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:06:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:06:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:07:09.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:07:09.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:07:09.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:07:09.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:07:09.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:07:09.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:07:09.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:07:09.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:07:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:07:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:07:09.775 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:07:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:07:39.638 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:07:39.645 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:07:39.646 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:07:39.683 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:07:39.683 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:07:39.683 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:07:39.683 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:07:39.683 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:07:39.683 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:07:39.683 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:07:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:07:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:07:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:07:49.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:07:49.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:07:49.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:07:49.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:07:49.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:07:49.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:07:49.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:07:49.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:07:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:07:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:07:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:08:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:08:09.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:08:09.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:08:09.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:08:09.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:08:09.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:08:09.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:08:09.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:08:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:08:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:08:09.761 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:08:39.633 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:08:39.634 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:09:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:09:39.632 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:09:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:09:39.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:09:39.674 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:09:39.674 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:09:39.674 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:09:39.674 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:09:39.674 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:09:39.674 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:09:39.674 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:09:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:09:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:09:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:09:49.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:09:49.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:09:49.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:09:49.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:09:49.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:09:49.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:09:49.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:09:49.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:09:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:09:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:09:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:10:09.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:10:09.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:10:09.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:10:09.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:10:09.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:10:09.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:10:09.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:10:09.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:10:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:10:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:10:09.754 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:10:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:10:39.636 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:10:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:10:39.643 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:10:39.677 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:10:39.677 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:10:39.677 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:10:39.677 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:10:39.677 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:10:39.677 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:10:39.677 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:10:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:10:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:10:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:10:49.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:10:49.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:10:49.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:10:49.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:10:49.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:10:49.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:10:49.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:10:49.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:10:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:10:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:10:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:11:09.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:11:09.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:11:09.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:11:09.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:11:09.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:11:09.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:11:09.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:11:09.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:11:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:11:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:11:09.754 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:11:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:11:39.637 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:11:39.649 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:11:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:11:39.682 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:11:39.682 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:11:39.682 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:11:39.682 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:11:39.682 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:11:39.682 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:11:39.682 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:11:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:11:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:11:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:11:49.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:11:49.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:11:49.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:11:49.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:11:49.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:11:49.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:11:49.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:11:49.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:11:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:11:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:11:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:12:09.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:12:09.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:12:09.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:12:09.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:12:09.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:12:09.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:12:09.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:12:09.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:12:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:12:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:12:09.757 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:12:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:12:39.640 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:12:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:12:39.648 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:12:39.682 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:12:39.682 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:12:39.682 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:12:39.682 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:12:39.682 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:12:39.682 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:12:39.682 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:12:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:12:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:12:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:12:49.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:12:49.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:12:49.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:12:49.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:12:49.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:12:49.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:12:49.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:12:49.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:12:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:12:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:12:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:13:09.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:13:09.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:13:09.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:13:09.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:13:09.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:13:09.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:13:09.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:13:09.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:13:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:13:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:13:09.759 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:13:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:13:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:13:39.649 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:13:39.650 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:13:39.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:13:39.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:13:39.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:13:39.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:13:39.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:13:39.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:13:39.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:13:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:13:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:13:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:13:49.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:13:49.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:13:49.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:13:49.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:13:49.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:13:49.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:13:49.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:13:49.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:13:49.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:13:49.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:13:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:14:09.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:14:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:14:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:14:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:14:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:14:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:14:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:14:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:14:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:14:09.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:14:09.771 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:14:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:14:39.643 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:14:39.652 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:14:39.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:14:39.690 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:14:39.690 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:14:39.690 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:14:39.690 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:14:39.690 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:14:39.690 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:14:39.690 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:14:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:14:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:14:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:14:49.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:14:49.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:14:49.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:14:49.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:14:49.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:14:49.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:14:49.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:14:49.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:14:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:14:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:14:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:15:09.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:15:09.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:15:09.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:15:09.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:15:09.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:15:09.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:15:09.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:15:09.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:15:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:15:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:15:09.771 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:15:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:15:39.644 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:15:39.653 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:15:39.654 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:15:39.695 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:15:39.695 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:15:39.695 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:15:39.695 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:15:39.695 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:15:39.695 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:15:39.695 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:15:39.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:15:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:15:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:15:49.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:15:49.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:15:49.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:15:49.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:15:49.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:15:49.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:15:49.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:15:49.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:15:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:15:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:15:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:16:09.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:16:09.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:16:09.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:16:09.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:16:09.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:16:09.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:16:09.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:16:09.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:16:09.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:16:09.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:16:09.772 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:16:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:16:39.644 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:16:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:16:39.655 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:16:39.694 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:16:39.694 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:16:39.694 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:16:39.694 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:16:39.694 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:16:39.694 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:16:39.694 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:16:39.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:16:39.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:16:39.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:16:49.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:16:49.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:16:49.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:16:49.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:16:49.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:16:49.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:16:49.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:16:49.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:16:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:16:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:16:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:17:09.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:17:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:17:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:17:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:17:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:17:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:17:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:17:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:17:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:17:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:17:09.771 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:17:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:17:39.643 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:17:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:17:39.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:17:39.691 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:17:39.691 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:17:39.691 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:17:39.691 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:17:39.691 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:17:39.691 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:17:39.691 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:17:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:17:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:17:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:17:49.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:17:49.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:17:49.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:17:49.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:17:49.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:17:49.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:17:49.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:17:49.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:17:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:17:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:17:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:18:09.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:18:09.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:18:09.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:18:09.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:18:09.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:18:09.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:18:09.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:18:09.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:18:09.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:18:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:18:09.768 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:18:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:18:39.641 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:18:39.648 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:18:39.648 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:18:39.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:18:39.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:18:39.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:18:39.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:18:39.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:18:39.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:18:39.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:18:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:18:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.051 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:18:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:18:49.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:18:49.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:18:49.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:18:49.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:18:49.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:18:49.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:18:49.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:18:49.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:18:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:18:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.087 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:18:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:19:09.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:19:09.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:19:09.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:19:09.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:19:09.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:19:09.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:19:09.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:19:09.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:19:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:19:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:19:09.774 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:19:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:19:39.636 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:19:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:19:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:19:39.682 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:19:39.682 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:19:39.682 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:19:39.682 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:19:39.682 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:19:39.682 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:19:39.682 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:19:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:19:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:19:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:19:49.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:19:49.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:19:49.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:19:49.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:19:49.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:19:49.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:19:49.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:19:49.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:19:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:19:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:19:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:20:09.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:20:09.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:20:09.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:20:09.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:20:09.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:20:09.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:20:09.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:20:09.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:20:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:20:09.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:20:09.767 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:20:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:20:39.640 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:20:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:20:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:20:39.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:20:39.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:20:39.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:20:39.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:20:39.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:20:39.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:20:39.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:20:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:20:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:20:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:20:49.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:20:49.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:20:49.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:20:49.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:20:49.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:20:49.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:20:49.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:20:49.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:20:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:20:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:20:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:21:09.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:21:09.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:21:09.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:21:09.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:21:09.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:21:09.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:21:09.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:21:09.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:21:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:21:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:21:09.755 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:21:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:21:39.638 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:21:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:21:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:21:39.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:21:39.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:21:39.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:21:39.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:21:39.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:21:39.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:21:39.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:21:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:21:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:21:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:21:49.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:21:49.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:21:49.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:21:49.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:21:49.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:21:49.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:21:49.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:21:49.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:21:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:21:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:21:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:22:09.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:22:09.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:22:09.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:22:09.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:22:09.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:22:09.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:22:09.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:22:09.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:22:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:22:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:22:09.759 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:22:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:22:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:22:39.649 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:22:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:22:39.683 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:22:39.683 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:22:39.683 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:22:39.683 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:22:39.683 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:22:39.683 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:22:39.683 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:22:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:22:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:22:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:22:49.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:22:49.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:22:49.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:22:49.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:22:49.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:22:49.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:22:49.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:22:49.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:22:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:22:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:22:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:23:09.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:23:09.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:23:09.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:23:09.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:23:09.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:23:09.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:23:09.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:23:09.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:23:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:23:09.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:23:09.764 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:23:39.645 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:23:39.646 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:23:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:23:39.655 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:23:39.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:23:39.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:23:39.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:23:39.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:23:39.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:23:39.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:23:39.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:23:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:23:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:23:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:23:49.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:23:49.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:23:49.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:23:49.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:23:49.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:23:49.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:23:49.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:23:49.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:23:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:23:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:23:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:24:09.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:24:09.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:24:09.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:24:09.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:24:09.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:24:09.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:24:09.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:24:09.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:24:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:24:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:24:09.766 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:24:39.648 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:24:39.650 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:24:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:24:39.657 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:24:39.705 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:24:39.705 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:24:39.705 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:24:39.705 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:24:39.705 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:24:39.705 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:24:39.705 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:24:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:24:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.05 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:24:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:24:49.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:24:49.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:24:49.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:24:49.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:24:49.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:24:49.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:24:49.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:24:49.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:24:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:24:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:24:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:25:09.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:25:09.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:25:09.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:25:09.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:25:09.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:25:09.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:25:09.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:25:09.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:25:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:25:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:25:09.781 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:25:39.652 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:25:39.653 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:25:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:25:39.662 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:25:39.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:25:39.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:25:39.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:25:39.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:25:39.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:25:39.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:25:39.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:25:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:25:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:25:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:25:49.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:25:49.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:25:49.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:25:49.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:25:49.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:25:49.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:25:49.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:25:49.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:25:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:25:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:25:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:26:09.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:26:09.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:26:09.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:26:09.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:26:09.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:26:09.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:26:09.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:26:09.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:26:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:26:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:26:09.783 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:26:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:26:39.655 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:26:39.661 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:26:39.662 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:26:39.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:26:39.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:26:39.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:26:39.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:26:39.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:26:39.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:26:39.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:26:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:26:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:26:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:26:49.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:26:49.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:26:49.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:26:49.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:26:49.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:26:49.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:26:49.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:26:49.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:26:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:26:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:26:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:27:09.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:27:09.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:27:09.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:27:09.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:27:09.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:27:09.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:27:09.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:27:09.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:27:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:27:09.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:27:09.779 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:27:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:27:39.652 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:27:39.658 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:27:39.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:27:39.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:27:39.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:27:39.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:27:39.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:27:39.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:27:39.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:27:39.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:27:39.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:27:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:27:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:27:49.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:27:49.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:27:49.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:27:49.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:27:49.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:27:49.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:27:49.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:27:49.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:27:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:27:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:27:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:28:09.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:28:09.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:28:09.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:28:09.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:28:09.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:28:09.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:28:09.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:28:09.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:28:09.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:28:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:28:09.780 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:28:39.652 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:28:39.652 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:28:39.660 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:28:39.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:28:39.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:28:39.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:28:39.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:28:39.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:28:39.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:28:39.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:28:39.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:28:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:28:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:28:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:28:49.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:28:49.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:28:49.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:28:49.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:28:49.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:28:49.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:28:49.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:28:49.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:28:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:28:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:28:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:29:09.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:29:09.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:29:09.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:29:09.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:29:09.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:29:09.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:29:09.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:29:09.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:29:09.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:29:09.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:29:09.775 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:29:39.656 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:29:39.657 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:29:39.665 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:29:39.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:29:39.700 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:29:39.700 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:29:39.700 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:29:39.700 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:29:39.700 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:29:39.700 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:29:39.700 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:29:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:29:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:29:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:29:49.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:29:49.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:29:49.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:29:49.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:29:49.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:29:49.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:29:49.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:29:49.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:29:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:29:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:29:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:30:09.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:30:09.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:30:09.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:30:09.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:30:09.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:30:09.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:30:09.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:30:09.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:30:09.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:30:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:30:09.780 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:30:39.652 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:30:39.653 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:30:39.660 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:30:39.661 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:30:39.700 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:30:39.700 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:30:39.700 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:30:39.700 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:30:39.700 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:30:39.700 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:30:39.700 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:30:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:30:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:30:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:30:49.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:30:49.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:30:49.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:30:49.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:30:49.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:30:49.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:30:49.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:30:49.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:30:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:30:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.087 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:30:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:31:09.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:31:09.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:31:09.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:31:09.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:31:09.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:31:09.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:31:09.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:31:09.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:31:09.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:31:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:31:09.784 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:31:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:31:39.656 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:31:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:31:39.671 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:31:39.704 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:31:39.704 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:31:39.704 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:31:39.704 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:31:39.704 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:31:39.704 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:31:39.704 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:31:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:31:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:31:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:31:49.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:31:49.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:31:49.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:31:49.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:31:49.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:31:49.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:31:49.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:31:49.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:31:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:31:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:31:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:32:09.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:32:09.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:32:09.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:32:09.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:32:09.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:32:09.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:32:09.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:32:09.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:32:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:32:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:32:09.778 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:32:39.660 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:32:39.661 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:32:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:32:39.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:32:39.708 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:32:39.708 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:32:39.708 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:32:39.708 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:32:39.708 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:32:39.708 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:32:39.708 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:32:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:32:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:32:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:32:49.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:32:49.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:32:49.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:32:49.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:32:49.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:32:49.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:32:49.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:32:49.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:32:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:32:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:32:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:33:09.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:33:09.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:33:09.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:33:09.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:33:09.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:33:09.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:33:09.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:33:09.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:33:09.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:33:09.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:33:09.784 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:33:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:33:39.656 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:33:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:33:39.663 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:33:39.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:33:39.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:33:39.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:33:39.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:33:39.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:33:39.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:33:39.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:33:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:33:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:33:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:33:49.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:33:49.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:33:49.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:33:49.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:33:49.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:33:49.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:33:49.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:33:49.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:33:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:33:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:33:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:34:09.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:34:09.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:34:09.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:34:09.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:34:09.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:34:09.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:34:09.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:34:09.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:34:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:34:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:34:09.778 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:34:39.660 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:34:39.660 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:34:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:34:39.668 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:34:39.701 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:34:39.701 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:34:39.701 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:34:39.701 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:34:39.701 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:34:39.701 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:34:39.701 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:34:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:34:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:34:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:34:49.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:34:49.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:34:49.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:34:49.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:34:49.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:34:49.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:34:49.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:34:49.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:34:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:34:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:34:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:35:09.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:35:09.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:35:09.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:35:09.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:35:09.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:35:09.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:35:09.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:35:09.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:35:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:35:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:35:09.777 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:35:39.658 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:35:39.659 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:35:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:35:39.668 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:35:39.701 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:35:39.701 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:35:39.701 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:35:39.701 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:35:39.701 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:35:39.701 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:35:39.701 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:35:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:35:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:35:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:35:49.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:35:49.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:35:49.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:35:49.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:35:49.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:35:49.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:35:49.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:35:49.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:35:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:35:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:35:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:36:09.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:36:09.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:36:09.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:36:09.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:36:09.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:36:09.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:36:09.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:36:09.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:36:09.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:36:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:36:09.778 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:36:39.660 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:36:39.661 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:36:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:36:39.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:36:39.705 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:36:39.705 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:36:39.705 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:36:39.705 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:36:39.705 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:36:39.705 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:36:39.705 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:36:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:36:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:36:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:36:49.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:36:49.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:36:49.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:36:49.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:36:49.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:36:49.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:36:49.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:36:49.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:36:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:36:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:36:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:37:09.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:37:09.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:37:09.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:37:09.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:37:09.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:37:09.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:37:09.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:37:09.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:37:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:37:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:37:09.782 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:37:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:37:39.665 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:37:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:37:39.672 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:37:39.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:37:39.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:37:39.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:37:39.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:37:39.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:37:39.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:37:39.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:37:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:37:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:37:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:37:49.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:37:49.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:37:49.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:37:49.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:37:49.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:37:49.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:37:49.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:37:49.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:37:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:37:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:37:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:38:09.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:38:09.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:38:09.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:38:09.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:38:09.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:38:09.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:38:09.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:38:09.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:38:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:38:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:38:09.785 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:38:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:38:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:38:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:38:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:38:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:38:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:38:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:38:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:38:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:38:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:38:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:38:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:38:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:38:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:38:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:38:49.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:38:49.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:38:49.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:38:49.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:38:49.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:38:49.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:38:49.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:38:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:38:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:38:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:39:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:39:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:39:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:39:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:39:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:39:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:39:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:39:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:39:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:39:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:39:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:39:39.665 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:39:39.666 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:39:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:39:39.674 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:39:39.707 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:39:39.707 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:39:39.707 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:39:39.707 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:39:39.707 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:39:39.707 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:39:39.707 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:39:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:39:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:39:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:39:49.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:39:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:39:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:39:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:39:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:39:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:39:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:39:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:39:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:39:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.094 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:39:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:40:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:40:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:40:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:40:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:40:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:40:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:40:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:40:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:40:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:40:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.128 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:40:09.802 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:40:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:40:39.665 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:40:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:40:39.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:40:39.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:40:39.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:40:39.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:40:39.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:40:39.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:40:39.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:40:39.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:40:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:40:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:40:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:40:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:40:49.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:40:49.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:40:49.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:40:49.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:40:49.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:40:49.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:40:49.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:40:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:40:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:40:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:41:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:41:09.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:41:09.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:41:09.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:41:09.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:41:09.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:41:09.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:41:09.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:41:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:41:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:41:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:41:39.665 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:41:39.665 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:41:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:41:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:41:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:41:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:41:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:41:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:41:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:41:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:41:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:41:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:41:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:41:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:41:49.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:41:49.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:41:49.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:41:49.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:41:49.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:41:49.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:41:49.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:41:49.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:41:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:41:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:41:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:42:09.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:42:09.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:42:09.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:42:09.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:42:09.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:42:09.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:42:09.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:42:09.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:42:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:42:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:42:09.786 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:42:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:42:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:42:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:42:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:42:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:42:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:42:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:42:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:42:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:42:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:42:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:42:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:42:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:42:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:42:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:42:49.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:42:49.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:42:49.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:42:49.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:42:49.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:42:49.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:42:49.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:42:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:42:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:42:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:43:09.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:43:09.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:43:09.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:43:09.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:43:09.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:43:09.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:43:09.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:43:09.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:43:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:43:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:43:09.786 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:43:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:43:39.668 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:43:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:43:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:43:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:43:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:43:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:43:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:43:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:43:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:43:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:43:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:43:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.051 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:43:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:43:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:43:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:43:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:43:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:43:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:43:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:43:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:43:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:43:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:43:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.092 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:43:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:44:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:44:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:44:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:44:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:44:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:44:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:44:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:44:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:44:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:44:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:44:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:44:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:44:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:44:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:44:39.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:44:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:44:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:44:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:44:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:44:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:44:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:44:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:44:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:44:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:44:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:44:49.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:44:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:44:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:44:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:44:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:44:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:44:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:44:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:44:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:44:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:44:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:45:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:45:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:45:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:45:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:45:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:45:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:45:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:45:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:45:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:45:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:45:09.791 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:45:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:45:39.663 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:45:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:45:39.672 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:45:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:45:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:45:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:45:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:45:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:45:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:45:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:45:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:45:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.046 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:45:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:45:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:45:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:45:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:45:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:45:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:45:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:45:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:45:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:45:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:45:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:45:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:46:09.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:46:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:46:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:46:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:46:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:46:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:46:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:46:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:46:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:46:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:46:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:46:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:46:39.663 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:46:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:46:39.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:46:39.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:46:39.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:46:39.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:46:39.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:46:39.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:46:39.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:46:39.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:46:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:46:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:46:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:46:49.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:46:49.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:46:49.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:46:49.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:46:49.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:46:49.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:46:49.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:46:49.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:46:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:46:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:46:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:47:09.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:47:09.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:47:09.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:47:09.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:47:09.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:47:09.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:47:09.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:47:09.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:47:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:47:09.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:47:09.777 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:47:39.659 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:47:39.660 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:47:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:47:39.668 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:47:39.702 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:47:39.702 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:47:39.702 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:47:39.702 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:47:39.702 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:47:39.702 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:47:39.702 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:47:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:47:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:47:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:47:49.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:47:49.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:47:49.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:47:49.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:47:49.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:47:49.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:47:49.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:47:49.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:47:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:47:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:47:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:48:09.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:48:09.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:48:09.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:48:09.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:48:09.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:48:09.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:48:09.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:48:09.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:48:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:48:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:48:09.782 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:48:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:48:39.655 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:48:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:48:39.663 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:48:39.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:48:39.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:48:39.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:48:39.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:48:39.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:48:39.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:48:39.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:48:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:48:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:48:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:48:49.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:48:49.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:48:49.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:48:49.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:48:49.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:48:49.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:48:49.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:48:49.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:48:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:48:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:48:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:49:09.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:49:09.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:49:09.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:49:09.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:49:09.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:49:09.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:49:09.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:49:09.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:49:09.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:49:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:49:09.780 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:49:39.651 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:49:39.652 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:49:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:49:39.662 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:49:39.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:49:39.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:49:39.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:49:39.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:49:39.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:49:39.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:49:39.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:49:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:49:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:49:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:49:49.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:49:49.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:49:49.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:49:49.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:49:49.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:49:49.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:49:49.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:49:49.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:49:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:49:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:49:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:50:09.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:50:09.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:50:09.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:50:09.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:50:09.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:50:09.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:50:09.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:50:09.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:50:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:50:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:50:09.776 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:50:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:50:39.648 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:50:39.656 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:50:39.657 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:50:39.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:50:39.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:50:39.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:50:39.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:50:39.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:50:39.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:50:39.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:50:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:50:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:50:39.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:50:49.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:50:49.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:50:49.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:50:49.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:50:49.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:50:49.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:50:49.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:50:49.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:50:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:50:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:50:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:51:09.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:51:09.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:51:09.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:51:09.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:51:09.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:51:09.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:51:09.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:51:09.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:51:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:51:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:51:09.769 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:51:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:51:39.651 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:51:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:51:39.662 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:51:39.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:51:39.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:51:39.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:51:39.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:51:39.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:51:39.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:51:39.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:51:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:51:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:51:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:51:49.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:51:49.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:51:49.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:51:49.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:51:49.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:51:49.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:51:49.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:51:49.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:51:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:51:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:51:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:52:09.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:52:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:52:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:52:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:52:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:52:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:52:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:52:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:52:09.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:52:09.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:52:09.771 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:52:39.653 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:52:39.654 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:52:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:52:39.664 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:52:39.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:52:39.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:52:39.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:52:39.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:52:39.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:52:39.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:52:39.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:52:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:52:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:52:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:52:49.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:52:49.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:52:49.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:52:49.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:52:49.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:52:49.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:52:49.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:52:49.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:52:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:52:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:52:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:53:09.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:53:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:53:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:53:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:53:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:53:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:53:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:53:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:53:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:53:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:53:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:53:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:53:39.655 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:53:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:53:39.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:53:39.705 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:53:39.705 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:53:39.705 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:53:39.705 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:53:39.705 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:53:39.705 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:53:39.705 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:53:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:53:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:53:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:53:49.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:53:49.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:53:49.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:53:49.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:53:49.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:53:49.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:53:49.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:53:49.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:53:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:53:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:53:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:54:09.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:54:09.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:54:09.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:54:09.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:54:09.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:54:09.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:54:09.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:54:09.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:54:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:54:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:54:09.786 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:54:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:54:39.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:54:39.691 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:54:39.691 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:54:39.691 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:54:39.691 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:54:39.691 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:54:39.691 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:54:39.691 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:54:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:54:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:54:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:54:49.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:54:49.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:54:49.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:54:49.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:54:49.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:54:49.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:54:49.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:54:49.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:54:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:54:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:54:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:55:09.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:55:09.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:55:09.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:55:09.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:55:09.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:55:09.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:55:09.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:55:09.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:55:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:55:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:55:09.766 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:55:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:55:39.659 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:55:39.665 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:55:39.666 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:55:39.706 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:55:39.706 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:55:39.706 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:55:39.706 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:55:39.706 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:55:39.706 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:55:39.706 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:55:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:55:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:55:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:55:49.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:55:49.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:55:49.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:55:49.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:55:49.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:55:49.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:55:49.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:55:49.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:55:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:55:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:55:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:56:09.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:56:09.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:56:09.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:56:09.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:56:09.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:56:09.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:56:09.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:56:09.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:56:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:56:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:56:09.782 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:56:39.653 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:56:39.654 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:56:39.661 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:56:39.661 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:56:39.695 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:56:39.695 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:56:39.695 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:56:39.695 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:56:39.695 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:56:39.695 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:56:39.695 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:56:39.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:56:39.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:56:39.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:56:49.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:56:49.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:56:49.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:56:49.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:56:49.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:56:49.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:56:49.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:56:49.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:56:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:56:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:56:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:57:09.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:57:09.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:57:09.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:57:09.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:57:09.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:57:09.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:57:09.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:57:09.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:57:09.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:57:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:57:09.775 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:57:39.656 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:57:39.657 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:57:39.665 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:57:39.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:57:39.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:57:39.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:57:39.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:57:39.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:57:39.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:57:39.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:57:39.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:57:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:57:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:57:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:57:49.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:57:49.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:57:49.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:57:49.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:57:49.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:57:49.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:57:49.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:57:49.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:57:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:57:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:57:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:58:09.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:58:09.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:58:09.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:58:09.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:58:09.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:58:09.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:58:09.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:58:09.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:58:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:58:09.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:58:09.774 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:58:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:58:39.656 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:58:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:58:39.663 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:58:39.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:58:39.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:58:39.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:58:39.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:58:39.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:58:39.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:58:39.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:58:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:58:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:58:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:58:49.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:58:49.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:58:49.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:58:49.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:58:49.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:58:49.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:58:49.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:58:49.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:58:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:58:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:58:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:59:09.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:59:09.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:59:09.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:59:09.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:59:09.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:59:09.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:59:09.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:59:09.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:59:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:59:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 16:59:09.773 139328 ERROR oslo_service.periodic_task 
>2018-06-24 16:59:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:59:39.656 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 16:59:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 16:59:39.663 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:59:39.695 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:59:39.695 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:59:39.695 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:59:39.695 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:59:39.695 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:59:39.695 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:59:39.695 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:59:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:59:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:59:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 16:59:49.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 16:59:49.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 16:59:49.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 16:59:49.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 16:59:49.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 16:59:49.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 16:59:49.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 16:59:49.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 16:59:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 16:59:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 16:59:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:00:09.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:00:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:00:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:00:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:00:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:00:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:00:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:00:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:00:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:00:09.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:00:09.771 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:00:39.652 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:00:39.653 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:00:39.665 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:00:39.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:00:39.701 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:00:39.701 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:00:39.701 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:00:39.701 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:00:39.701 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:00:39.701 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:00:39.701 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:00:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:00:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:00:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:00:49.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:00:49.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:00:49.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:00:49.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:00:49.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:00:49.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:00:49.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:00:49.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:00:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:00:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:00:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:01:09.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:01:09.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:01:09.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:01:09.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:01:09.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:01:09.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:01:09.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:01:09.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:01:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:01:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:01:09.779 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:01:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:01:39.651 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:01:39.658 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:01:39.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:01:39.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:01:39.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:01:39.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:01:39.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:01:39.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:01:39.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:01:39.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:01:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:01:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:01:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:01:49.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:01:49.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:01:49.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:01:49.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:01:49.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:01:49.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:01:49.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:01:49.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:01:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:01:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:01:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:02:09.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:02:09.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:02:09.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:02:09.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:02:09.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:02:09.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:02:09.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:02:09.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:02:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:02:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:02:09.773 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:02:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:02:39.656 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:02:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:02:39.664 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:02:39.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:02:39.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:02:39.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:02:39.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:02:39.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:02:39.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:02:39.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:02:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:02:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:02:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:02:49.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:02:49.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:02:49.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:02:49.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:02:49.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:02:49.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:02:49.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:02:49.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:02:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:02:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:02:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:03:09.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:03:09.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:03:09.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:03:09.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:03:09.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:03:09.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:03:09.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:03:09.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:03:09.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:03:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:03:09.773 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:03:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:03:39.656 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:03:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:03:39.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:03:39.700 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:03:39.700 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:03:39.700 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:03:39.700 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:03:39.700 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:03:39.700 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:03:39.700 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:03:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:03:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:03:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:03:49.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:03:49.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:03:49.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:03:49.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:03:49.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:03:49.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:03:49.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:03:49.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:03:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:03:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:03:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:04:09.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:04:09.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:04:09.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:04:09.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:04:09.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:04:09.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:04:09.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:04:09.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:04:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:04:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:04:09.781 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:04:39.652 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:04:39.653 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:04:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:04:39.662 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:04:39.695 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:04:39.695 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:04:39.695 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:04:39.695 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:04:39.695 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:04:39.695 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:04:39.695 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:04:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:04:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:04:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:04:49.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:04:49.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:04:49.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:04:49.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:04:49.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:04:49.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:04:49.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:04:49.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:04:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:04:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:04:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:05:09.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:05:09.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:05:09.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:05:09.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:05:09.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:05:09.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:05:09.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:05:09.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:05:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:05:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:05:09.779 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:05:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:05:39.651 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:05:39.659 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:05:39.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:05:39.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:05:39.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:05:39.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:05:39.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:05:39.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:05:39.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:05:39.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:05:39.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:05:39.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:05:39.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:05:49.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:05:49.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:05:49.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:05:49.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:05:49.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:05:49.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:05:49.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:05:49.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:05:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:05:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:05:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:06:09.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:06:09.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:06:09.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:06:09.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:06:09.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:06:09.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:06:09.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:06:09.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:06:09.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:06:09.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:06:09.774 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:06:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:06:39.647 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:06:39.656 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:06:39.657 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:06:39.691 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:06:39.691 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:06:39.691 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:06:39.691 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:06:39.691 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:06:39.691 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:06:39.691 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:06:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:06:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:06:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:06:49.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:06:49.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:06:49.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:06:49.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:06:49.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:06:49.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:06:49.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:06:49.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:06:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:06:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:06:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:07:09.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:07:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:07:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:07:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:07:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:07:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:07:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:07:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:07:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:07:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:07:09.770 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:07:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:07:39.643 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:07:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:07:39.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:07:39.684 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:07:39.684 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:07:39.684 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:07:39.684 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:07:39.684 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:07:39.684 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:07:39.684 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:07:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:07:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:07:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:07:49.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:07:49.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:07:49.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:07:49.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:07:49.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:07:49.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:07:49.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:07:49.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:07:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:07:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:07:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:08:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:08:09.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:08:09.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:08:09.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:08:09.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:08:09.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:08:09.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:08:09.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:08:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:08:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:08:09.762 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:08:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:08:39.644 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:08:39.653 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:08:39.654 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:08:39.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:08:39.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:08:39.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:08:39.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:08:39.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:08:39.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:08:39.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:08:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:08:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:08:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:08:49.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:08:49.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:08:49.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:08:49.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:08:49.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:08:49.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:08:49.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:08:49.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:08:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:08:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:08:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:09:09.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:09:09.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:09:09.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:09:09.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:09:09.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:09:09.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:09:09.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:09:09.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:09:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:09:09.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:09:09.769 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:09:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:09:39.641 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:09:39.658 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:09:39.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:09:39.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:09:39.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:09:39.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:09:39.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:09:39.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:09:39.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:09:39.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:09:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:09:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:09:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:09:49.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:09:49.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:09:49.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:09:49.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:09:49.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:09:49.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:09:49.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:09:49.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:09:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:09:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:09:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:10:09.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:10:09.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:10:09.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:10:09.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:10:09.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:10:09.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:10:09.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:10:09.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:10:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:10:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:10:09.775 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:10:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:10:39.638 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:10:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:10:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:10:39.688 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:10:39.688 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:10:39.688 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:10:39.688 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:10:39.688 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:10:39.688 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:10:39.688 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:10:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:10:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:10:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:10:49.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:10:49.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:10:49.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:10:49.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:10:49.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:10:49.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:10:49.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:10:49.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:10:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:10:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:10:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:11:09.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:11:09.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:11:09.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:11:09.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:11:09.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:11:09.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:11:09.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:11:09.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:11:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:11:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:11:09.767 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:11:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:11:39.639 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:11:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:11:39.648 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:11:39.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:11:39.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:11:39.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:11:39.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:11:39.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:11:39.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:11:39.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:11:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:11:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:11:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:11:49.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:11:49.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:11:49.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:11:49.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:11:49.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:11:49.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:11:49.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:11:49.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:11:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:11:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:11:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:12:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:12:09.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:12:09.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:12:09.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:12:09.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:12:09.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:12:09.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:12:09.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:12:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:12:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:12:09.763 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:12:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:12:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:12:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:12:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:12:39.680 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:12:39.680 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:12:39.680 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:12:39.680 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:12:39.680 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:12:39.680 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:12:39.680 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:12:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:12:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:12:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:12:49.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:12:49.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:12:49.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:12:49.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:12:49.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:12:49.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:12:49.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:12:49.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:12:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:12:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:12:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:13:09.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:13:09.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:13:09.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:13:09.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:13:09.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:13:09.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:13:09.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:13:09.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:13:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:13:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:13:09.758 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:13:39.630 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:13:39.631 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:13:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:13:39.638 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:13:39.671 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:13:39.671 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:13:39.671 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:13:39.671 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:13:39.671 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:13:39.671 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:13:39.671 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:13:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:13:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:13:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:13:49.674 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:13:49.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:13:49.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:13:49.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:13:49.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:13:49.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:13:49.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:13:49.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:13:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:13:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:13:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:14:09.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:14:09.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:14:09.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:14:09.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:14:09.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:14:09.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:14:09.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:14:09.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:14:09.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:14:09.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:14:09.747 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:14:39.628 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:14:39.629 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:14:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:14:39.636 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:14:39.671 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:14:39.671 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:14:39.671 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:14:39.671 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:14:39.671 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:14:39.671 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:14:39.671 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:14:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:14:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:14:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:14:49.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:14:49.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:14:49.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:14:49.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:14:49.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:14:49.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:14:49.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:14:49.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:14:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:14:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:14:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:15:09.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:15:09.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:15:09.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:15:09.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:15:09.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:15:09.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:15:09.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:15:09.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:15:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:15:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:15:09.751 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:15:39.633 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:15:39.634 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:15:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:15:39.642 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:15:39.676 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:15:39.676 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:15:39.676 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:15:39.676 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:15:39.676 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:15:39.676 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:15:39.676 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:15:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:15:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:15:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:15:49.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:15:49.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:15:49.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:15:49.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:15:49.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:15:49.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:15:49.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:15:49.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:15:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:15:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:15:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:16:09.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:16:09.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:16:09.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:16:09.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:16:09.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:16:09.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:16:09.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:16:09.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:16:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:16:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:16:09.752 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:16:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:16:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:16:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:16:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:16:39.680 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:16:39.680 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:16:39.680 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:16:39.680 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:16:39.680 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:16:39.680 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:16:39.680 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:16:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:16:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:16:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:16:49.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:16:49.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:16:49.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:16:49.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:16:49.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:16:49.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:16:49.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:16:49.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:16:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:16:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:16:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:17:09.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:17:09.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:17:09.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:17:09.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:17:09.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:17:09.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:17:09.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:17:09.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:17:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:17:09.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:17:09.760 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:17:39.632 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:17:39.632 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:17:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:17:39.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:17:39.673 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:17:39.673 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:17:39.673 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:17:39.673 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:17:39.673 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:17:39.673 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:17:39.673 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:17:39.675 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:17:39.675 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:17:39.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:17:49.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:17:49.708 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:17:49.708 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:17:49.708 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:17:49.708 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:17:49.708 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:17:49.708 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:17:49.708 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:17:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:17:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:17:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:18:09.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:18:09.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:18:09.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:18:09.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:18:09.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:18:09.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:18:09.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:18:09.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:18:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:18:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:18:09.750 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:18:39.632 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:18:39.633 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:18:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:18:39.641 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:18:39.675 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:18:39.675 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:18:39.675 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:18:39.675 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:18:39.675 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:18:39.675 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:18:39.675 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:18:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:18:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:18:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:18:49.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:18:49.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:18:49.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:18:49.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:18:49.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:18:49.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:18:49.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:18:49.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:18:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:18:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:18:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:19:09.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:19:09.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:19:09.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:19:09.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:19:09.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:19:09.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:19:09.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:19:09.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:19:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:19:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:19:09.766 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:19:39.627 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:19:39.628 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:19:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:19:39.636 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:19:39.671 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:19:39.671 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:19:39.671 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:19:39.671 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:19:39.671 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:19:39.671 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:19:39.671 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:19:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:19:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:19:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:19:49.674 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:19:49.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:19:49.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:19:49.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:19:49.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:19:49.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:19:49.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:19:49.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:19:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:19:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:19:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:20:09.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:20:09.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:20:09.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:20:09.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:20:09.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:20:09.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:20:09.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:20:09.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:20:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:20:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:20:09.755 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:20:39.627 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:20:39.628 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:20:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:20:39.636 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:20:39.681 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:20:39.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:20:39.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:20:39.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:20:39.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:20:39.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:20:39.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:20:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:20:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.049 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:20:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:20:49.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:20:49.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:20:49.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:20:49.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:20:49.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:20:49.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:20:49.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:20:49.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:20:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:20:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:20:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:21:09.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:21:09.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:21:09.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:21:09.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:21:09.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:21:09.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:21:09.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:21:09.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:21:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:21:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:21:09.757 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:21:39.629 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:21:39.630 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:21:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:21:39.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:21:39.675 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:21:39.675 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:21:39.675 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:21:39.675 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:21:39.675 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:21:39.675 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:21:39.675 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:21:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:21:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:21:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:21:49.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:21:49.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:21:49.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:21:49.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:21:49.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:21:49.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:21:49.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:21:49.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:21:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:21:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:21:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:22:09.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:22:09.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:22:09.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:22:09.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:22:09.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:22:09.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:22:09.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:22:09.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:22:09.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:22:09.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:22:09.749 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:22:39.630 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:22:39.631 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:22:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:22:39.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:22:39.672 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:22:39.672 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:22:39.672 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:22:39.672 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:22:39.672 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:22:39.672 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:22:39.672 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:22:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:22:39.675 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:22:39.675 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:22:49.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:22:49.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:22:49.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:22:49.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:22:49.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:22:49.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:22:49.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:22:49.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:22:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:22:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:22:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:23:09.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:23:09.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:23:09.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:23:09.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:23:09.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:23:09.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:23:09.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:23:09.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:23:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:23:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:23:09.751 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:23:39.633 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:23:39.634 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:23:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:23:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:23:39.678 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:23:39.678 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:23:39.678 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:23:39.678 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:23:39.678 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:23:39.678 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:23:39.678 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:23:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:23:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:23:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:23:49.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:23:49.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:23:49.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:23:49.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:23:49.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:23:49.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:23:49.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:23:49.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:23:49.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:23:49.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:23:49.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:24:09.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:24:09.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:24:09.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:24:09.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:24:09.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:24:09.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:24:09.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:24:09.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:24:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:24:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:24:09.762 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:24:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:24:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:25:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:25:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:25:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:25:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:25:39.684 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:25:39.684 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:25:39.684 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:25:39.684 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:25:39.684 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:25:39.684 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:25:39.684 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:25:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:25:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:25:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:25:49.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:25:49.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:25:49.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:25:49.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:25:49.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:25:49.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:25:49.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:25:49.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:25:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:25:49.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:25:49.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:26:09.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:26:09.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:26:09.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:26:09.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:26:09.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:26:09.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:26:09.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:26:09.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:26:09.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:26:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:26:09.768 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:26:39.629 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:26:39.631 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:26:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:26:39.640 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:26:39.674 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:26:39.674 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:26:39.674 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:26:39.674 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:26:39.674 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:26:39.674 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:26:39.674 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:26:39.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:26:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:26:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:26:49.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:26:49.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:26:49.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:26:49.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:26:49.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:26:49.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:26:49.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:26:49.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:26:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:26:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:26:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:27:09.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:27:09.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:27:09.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:27:09.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:27:09.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:27:09.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:27:09.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:27:09.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:27:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:27:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:27:09.759 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:27:39.630 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:27:39.631 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:27:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:27:39.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:27:39.674 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:27:39.674 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:27:39.674 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:27:39.674 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:27:39.674 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:27:39.674 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:27:39.674 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:27:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:27:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:27:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:27:49.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:27:49.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:27:49.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:27:49.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:27:49.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:27:49.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:27:49.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:27:49.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:27:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:27:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:27:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:28:09.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:28:09.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:28:09.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:28:09.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:28:09.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:28:09.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:28:09.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:28:09.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:28:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:28:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:28:09.753 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:28:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:28:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:28:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:28:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:28:39.677 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:28:39.677 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:28:39.677 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:28:39.677 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:28:39.677 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:28:39.677 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:28:39.677 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:28:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:28:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:28:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:28:49.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:28:49.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:28:49.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:28:49.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:28:49.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:28:49.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:28:49.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:28:49.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:28:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:28:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:28:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:29:09.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:29:09.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:29:09.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:29:09.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:29:09.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:29:09.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:29:09.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:29:09.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:29:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:29:09.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:29:09.769 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:29:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:29:39.632 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:29:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:29:39.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:29:39.673 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:29:39.673 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:29:39.673 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:29:39.673 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:29:39.673 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:29:39.673 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:29:39.673 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:29:39.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:29:39.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:29:39.676 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:29:49.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:29:49.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:29:49.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:29:49.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:29:49.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:29:49.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:29:49.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:29:49.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:29:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:29:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:29:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:30:09.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:30:09.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:30:09.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:30:09.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:30:09.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:30:09.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:30:09.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:30:09.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:30:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:30:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:30:09.757 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:30:39.628 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:30:39.629 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:30:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:30:39.637 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:30:39.670 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:30:39.670 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:30:39.670 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:30:39.670 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:30:39.670 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:30:39.670 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:30:39.670 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:30:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:30:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:30:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:30:49.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:30:49.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:30:49.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:30:49.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:30:49.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:30:49.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:30:49.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:30:49.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:30:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:30:49.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:30:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:31:09.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:31:09.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:31:09.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:31:09.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:31:09.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:31:09.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:31:09.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:31:09.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:31:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:31:09.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:31:09.749 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:31:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:31:39.632 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:31:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:31:39.640 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:31:39.676 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:31:39.676 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:31:39.676 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:31:39.676 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:31:39.676 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:31:39.676 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:31:39.676 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:31:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:31:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:31:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:31:49.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:31:49.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:31:49.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:31:49.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:31:49.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:31:49.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:31:49.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:31:49.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:31:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:31:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:31:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:32:09.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:32:09.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:32:09.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:32:09.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:32:09.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:32:09.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:32:09.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:32:09.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:32:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:32:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:32:09.752 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:32:39.633 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:32:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:32:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:32:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:32:39.680 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:32:39.680 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:32:39.680 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:32:39.680 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:32:39.680 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:32:39.680 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:32:39.680 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:32:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:32:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:32:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:32:49.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:32:49.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:32:49.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:32:49.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:32:49.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:32:49.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:32:49.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:32:49.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:32:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:32:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:32:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:33:09.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:33:09.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:33:09.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:33:09.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:33:09.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:33:09.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:33:09.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:33:09.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:33:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:33:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:33:09.756 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:33:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:33:39.638 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:33:39.645 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:33:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:33:39.678 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:33:39.678 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:33:39.678 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:33:39.678 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:33:39.678 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:33:39.678 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:33:39.678 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:33:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:33:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:33:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:33:49.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:33:49.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:33:49.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:33:49.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:33:49.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:33:49.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:33:49.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:33:49.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:33:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:33:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:33:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:34:09.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:34:09.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:34:09.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:34:09.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:34:09.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:34:09.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:34:09.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:34:09.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:34:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:34:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:34:09.752 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:34:39.633 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:34:39.634 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:34:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:34:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:34:39.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:34:39.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:34:39.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:34:39.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:34:39.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:34:39.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:34:39.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:34:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:34:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:34:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:34:49.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:34:49.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:34:49.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:34:49.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:34:49.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:34:49.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:34:49.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:34:49.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:34:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:34:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:34:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:35:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:35:09.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:35:09.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:35:09.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:35:09.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:35:09.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:35:09.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:35:09.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:35:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:35:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:35:09.762 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:35:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:35:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:35:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:35:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:35:39.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:35:39.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:35:39.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:35:39.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:35:39.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:35:39.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:35:39.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:35:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:35:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:35:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:35:49.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:35:49.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:35:49.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:35:49.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:35:49.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:35:49.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:35:49.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:35:49.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:35:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:35:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:35:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:36:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:36:09.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:36:09.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:36:09.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:36:09.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:36:09.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:36:09.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:36:09.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:36:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:36:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:36:09.759 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:36:39.631 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:36:39.632 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:36:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:36:39.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:36:39.674 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:36:39.674 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:36:39.674 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:36:39.674 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:36:39.674 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:36:39.674 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:36:39.674 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:36:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:36:39.677 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:36:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:36:49.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:36:49.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:36:49.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:36:49.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:36:49.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:36:49.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:36:49.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:36:49.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:36:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:36:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:36:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:37:09.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:37:09.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:37:09.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:37:09.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:37:09.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:37:09.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:37:09.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:37:09.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:37:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:37:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:37:09.758 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:37:39.629 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:37:39.630 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:37:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:37:39.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:37:39.675 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:37:39.675 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:37:39.675 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:37:39.675 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:37:39.675 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:37:39.675 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:37:39.675 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:37:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:37:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:37:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:37:49.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:37:49.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:37:49.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:37:49.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:37:49.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:37:49.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:37:49.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:37:49.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:37:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:37:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:37:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:38:09.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:38:09.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:38:09.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:38:09.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:38:09.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:38:09.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:38:09.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:38:09.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:38:09.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:38:09.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:38:09.748 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:38:39.629 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:38:39.630 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:38:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:38:39.637 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:38:39.670 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:38:39.670 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:38:39.670 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:38:39.670 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:38:39.670 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:38:39.670 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:38:39.670 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:38:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:38:39.673 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:38:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:38:49.674 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:38:49.708 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:38:49.708 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:38:49.708 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:38:49.708 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:38:49.708 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:38:49.708 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:38:49.708 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:38:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:38:49.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:38:49.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:39:09.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:39:09.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:39:09.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:39:09.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:39:09.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:39:09.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:39:09.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:39:09.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:39:09.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:39:09.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:39:09.746 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:39:39.627 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:39:39.628 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:39:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:39:39.635 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:39:39.669 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:39:39.669 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:39:39.669 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:39:39.669 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:39:39.669 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:39:39.669 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:39:39.669 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:39:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:39:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:39:39.672 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:39:49.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:39:49.705 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:39:49.705 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:39:49.705 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:39:49.705 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:39:49.705 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:39:49.705 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:39:49.705 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:39:49.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:39:49.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:39:49.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:40:09.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:40:09.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:40:09.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:40:09.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:40:09.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:40:09.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:40:09.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:40:09.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:40:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:40:09.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:40:09.745 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:40:39.626 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:40:39.628 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:40:39.645 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:40:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:40:39.682 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:40:39.682 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:40:39.682 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:40:39.682 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:40:39.682 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:40:39.682 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:40:39.682 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:40:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:40:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:40:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:40:49.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:40:49.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:40:49.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:40:49.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:40:49.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:40:49.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:40:49.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:40:49.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:40:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:40:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:40:49.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:41:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:41:09.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:41:09.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:41:09.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:41:09.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:41:09.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:41:09.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:41:09.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:41:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:41:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:41:09.758 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:41:39.629 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:41:39.630 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:41:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:41:39.639 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:41:39.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:41:39.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:41:39.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:41:39.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:41:39.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:41:39.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:41:39.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:41:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:41:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.05 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:41:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:41:49.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:41:49.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:41:49.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:41:49.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:41:49.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:41:49.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:41:49.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:41:49.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:41:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:41:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.087 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:41:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:42:09.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:42:09.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:42:09.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:42:09.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:42:09.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:42:09.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:42:09.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:42:09.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:42:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:42:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:42:09.762 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:42:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:42:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:42:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:42:39.642 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:42:39.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:42:39.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:42:39.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:42:39.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:42:39.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:42:39.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:42:39.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:42:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:42:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:42:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:42:49.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:42:49.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:42:49.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:42:49.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:42:49.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:42:49.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:42:49.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:42:49.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:42:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:42:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:42:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:43:09.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:43:09.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:43:09.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:43:09.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:43:09.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:43:09.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:43:09.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:43:09.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:43:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:43:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:43:09.752 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:43:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:43:39.634 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:43:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:43:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:43:39.680 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:43:39.680 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:43:39.680 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:43:39.680 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:43:39.680 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:43:39.680 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:43:39.680 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:43:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:43:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:43:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:43:49.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:43:49.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:43:49.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:43:49.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:43:49.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:43:49.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:43:49.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:43:49.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:43:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:43:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:43:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:44:09.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:44:09.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:44:09.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:44:09.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:44:09.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:44:09.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:44:09.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:44:09.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:44:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:44:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:44:09.771 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:44:39.633 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:44:39.634 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:44:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:44:39.642 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:44:39.678 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:44:39.678 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:44:39.678 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:44:39.678 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:44:39.678 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:44:39.678 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:44:39.678 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:44:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:44:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:44:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:44:49.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:44:49.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:44:49.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:44:49.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:44:49.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:44:49.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:44:49.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:44:49.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:44:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:44:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:44:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:45:09.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:45:09.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:45:09.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:45:09.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:45:09.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:45:09.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:45:09.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:45:09.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:45:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:45:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:45:09.755 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:45:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:45:39.638 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:45:39.648 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:45:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:45:39.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:45:39.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:45:39.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:45:39.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:45:39.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:45:39.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:45:39.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:45:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:45:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:45:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:45:49.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:45:49.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:45:49.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:45:49.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:45:49.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:45:49.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:45:49.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:45:49.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:45:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:45:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:45:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:46:09.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:46:09.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:46:09.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:46:09.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:46:09.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:46:09.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:46:09.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:46:09.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:46:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:46:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:46:09.769 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:46:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:46:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:46:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:46:39.650 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:46:39.685 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:46:39.685 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:46:39.685 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:46:39.685 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:46:39.685 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:46:39.685 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:46:39.685 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:46:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:46:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:46:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:46:49.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:46:49.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:46:49.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:46:49.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:46:49.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:46:49.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:46:49.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:46:49.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:46:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:46:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:46:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:47:09.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:47:09.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:47:09.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:47:09.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:47:09.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:47:09.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:47:09.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:47:09.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:47:09.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:47:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:47:09.761 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:47:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:47:39.643 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:47:39.653 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:47:39.654 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:47:39.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:47:39.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:47:39.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:47:39.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:47:39.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:47:39.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:47:39.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:47:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:47:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:47:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:47:49.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:47:49.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:47:49.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:47:49.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:47:49.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:47:49.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:47:49.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:47:49.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:47:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:47:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:47:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:48:09.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:48:09.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:48:09.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:48:09.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:48:09.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:48:09.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:48:09.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:48:09.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:48:09.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:48:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:48:09.768 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:48:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:48:39.640 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:48:39.649 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:48:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:48:39.684 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:48:39.684 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:48:39.684 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:48:39.684 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:48:39.684 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:48:39.684 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:48:39.684 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:48:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:48:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:48:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:48:49.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:48:49.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:48:49.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:48:49.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:48:49.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:48:49.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:48:49.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:48:49.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:48:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:48:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:48:49.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:49:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:49:09.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:49:09.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:49:09.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:49:09.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:49:09.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:49:09.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:49:09.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:49:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:49:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:49:09.758 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:49:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:49:39.641 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:49:39.651 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:49:39.652 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:49:39.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:49:39.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:49:39.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:49:39.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:49:39.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:49:39.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:49:39.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:49:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:49:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:49:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:49:49.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:49:49.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:49:49.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:49:49.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:49:49.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:49:49.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:49:49.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:49:49.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:49:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:49:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:49:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:50:09.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:50:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:50:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:50:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:50:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:50:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:50:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:50:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:50:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:50:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:50:09.771 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:50:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:50:39.643 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:50:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:50:39.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:50:39.685 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:50:39.685 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:50:39.685 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:50:39.685 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:50:39.685 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:50:39.685 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:50:39.685 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:50:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:50:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:50:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:50:49.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:50:49.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:50:49.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:50:49.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:50:49.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:50:49.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:50:49.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:50:49.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:50:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:50:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:50:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:51:09.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:51:09.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:51:09.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:51:09.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:51:09.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:51:09.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:51:09.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:51:09.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:51:09.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:51:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:51:09.765 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:51:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:51:39.648 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:51:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:51:39.654 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:51:39.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:51:39.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:51:39.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:51:39.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:51:39.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:51:39.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:51:39.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:51:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:51:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:51:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:51:49.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:51:49.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:51:49.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:51:49.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:51:49.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:51:49.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:51:49.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:51:49.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:51:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:51:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:51:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:52:09.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:52:09.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:52:09.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:52:09.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:52:09.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:52:09.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:52:09.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:52:09.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:52:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:52:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:52:09.766 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:52:39.648 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:52:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:52:39.688 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:52:39.688 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:52:39.688 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:52:39.688 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:52:39.688 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:52:39.688 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:52:39.688 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:52:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:52:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:52:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:52:49.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:52:49.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:52:49.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:52:49.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:52:49.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:52:49.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:52:49.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:52:49.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:52:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:52:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:52:49.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:53:09.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:53:09.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:53:09.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:53:09.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:53:09.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:53:09.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:53:09.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:53:09.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:53:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:53:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:53:09.767 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:53:39.648 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:53:39.649 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:53:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:53:39.657 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:53:39.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:53:39.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:53:39.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:53:39.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:53:39.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:53:39.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:53:39.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:53:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:53:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:53:39.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:53:49.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:53:49.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:53:49.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:53:49.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:53:49.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:53:49.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:53:49.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:53:49.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:53:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:53:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:53:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:54:09.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:54:09.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:54:09.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:54:09.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:54:09.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:54:09.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:54:09.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:54:09.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:54:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:54:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:54:09.776 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:54:39.648 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:54:39.649 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:54:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:54:39.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:54:39.688 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:54:39.688 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:54:39.688 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:54:39.688 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:54:39.688 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:54:39.688 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:54:39.688 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:54:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:54:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:54:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:54:49.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:54:49.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:54:49.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:54:49.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:54:49.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:54:49.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:54:49.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:54:49.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:54:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:54:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:54:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:55:09.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:55:09.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:55:09.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:55:09.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:55:09.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:55:09.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:55:09.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:55:09.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:55:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:55:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:55:09.764 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:55:39.645 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:55:39.646 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:55:39.653 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:55:39.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:55:39.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:55:39.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:55:39.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:55:39.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:55:39.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:55:39.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:55:39.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:55:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:55:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.046 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:55:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:55:49.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:55:49.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:55:49.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:55:49.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:55:49.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:55:49.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:55:49.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:55:49.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:55:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:55:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:55:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:56:09.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:56:09.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:56:09.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:56:09.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:56:09.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:56:09.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:56:09.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:56:09.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:56:09.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:56:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:56:09.778 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:56:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:56:39.651 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:56:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:56:39.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:56:39.691 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:56:39.691 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:56:39.691 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:56:39.691 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:56:39.691 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:56:39.691 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:56:39.691 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:56:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:56:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:56:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:56:49.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:56:49.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:56:49.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:56:49.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:56:49.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:56:49.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:56:49.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:56:49.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:56:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:56:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:56:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:57:09.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:57:09.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:57:09.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:57:09.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:57:09.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:57:09.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:57:09.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:57:09.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:57:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:57:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:57:09.766 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:57:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:57:39.648 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:57:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:57:39.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:57:39.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:57:39.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:57:39.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:57:39.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:57:39.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:57:39.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:57:39.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:57:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:57:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:57:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:57:49.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:57:49.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:57:49.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:57:49.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:57:49.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:57:49.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:57:49.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:57:49.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:57:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:57:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:57:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:58:09.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:58:09.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:58:09.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:58:09.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:58:09.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:58:09.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:58:09.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:58:09.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:58:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:58:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:58:09.761 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:58:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:58:39.643 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:58:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:58:39.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:58:39.686 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:58:39.686 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:58:39.686 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:58:39.686 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:58:39.686 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:58:39.686 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:58:39.686 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:58:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:58:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:58:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:58:49.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:58:49.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:58:49.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:58:49.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:58:49.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:58:49.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:58:49.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:58:49.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:58:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:58:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:58:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:59:09.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:59:09.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:59:09.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:59:09.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:59:09.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:59:09.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:59:09.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:59:09.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:59:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:59:09.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 17:59:09.769 139328 ERROR oslo_service.periodic_task 
>2018-06-24 17:59:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:59:39.641 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 17:59:39.648 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 17:59:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:59:39.683 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:59:39.683 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:59:39.683 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:59:39.683 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:59:39.683 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:59:39.683 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:59:39.683 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:59:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:59:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:59:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 17:59:49.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 17:59:49.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 17:59:49.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 17:59:49.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 17:59:49.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 17:59:49.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 17:59:49.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 17:59:49.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 17:59:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 17:59:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 17:59:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:00:09.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:00:09.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:00:09.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:00:09.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:00:09.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:00:09.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:00:09.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:00:09.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:00:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:00:09.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:00:09.779 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:00:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:00:39.641 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:00:39.651 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:00:39.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:00:39.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:00:39.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:00:39.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:00:39.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:00:39.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:00:39.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:00:39.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:00:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:00:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:00:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:00:49.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:00:49.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:00:49.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:00:49.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:00:49.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:00:49.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:00:49.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:00:49.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:00:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:00:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:00:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:01:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:01:09.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:01:09.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:01:09.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:01:09.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:01:09.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:01:09.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:01:09.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:01:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:01:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:01:09.766 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:01:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:01:39.639 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:01:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:01:39.650 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:01:39.684 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:01:39.684 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:01:39.684 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:01:39.684 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:01:39.684 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:01:39.684 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:01:39.684 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:01:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:01:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:01:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:01:49.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:01:49.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:01:49.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:01:49.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:01:49.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:01:49.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:01:49.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:01:49.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:01:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:01:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:01:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:02:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:02:09.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:02:09.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:02:09.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:02:09.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:02:09.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:02:09.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:02:09.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:02:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:02:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:02:09.759 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:02:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:02:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:02:39.648 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:02:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:02:39.682 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:02:39.682 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:02:39.682 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:02:39.682 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:02:39.682 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:02:39.682 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:02:39.682 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:02:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:02:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:02:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:02:49.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:02:49.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:02:49.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:02:49.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:02:49.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:02:49.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:02:49.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:02:49.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:02:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:02:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:02:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:03:09.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:03:09.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:03:09.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:03:09.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:03:09.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:03:09.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:03:09.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:03:09.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:03:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:03:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:03:09.758 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:03:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:03:39.641 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:03:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:03:39.655 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:03:39.687 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:03:39.687 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:03:39.687 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:03:39.687 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:03:39.687 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:03:39.687 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:03:39.687 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:03:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:03:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:03:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:03:49.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:03:49.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:03:49.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:03:49.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:03:49.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:03:49.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:03:49.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:03:49.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:03:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:03:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:03:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:04:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:04:09.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:04:09.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:04:09.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:04:09.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:04:09.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:04:09.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:04:09.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:04:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:04:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:04:09.763 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:04:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:04:39.636 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:04:39.649 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:04:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:04:39.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:04:39.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:04:39.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:04:39.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:04:39.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:04:39.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:04:39.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:04:39.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:04:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.053 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:04:39.703 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:04:49.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:04:49.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:04:49.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:04:49.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:04:49.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:04:49.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:04:49.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:04:49.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:04:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:04:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.092 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:04:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:05:09.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:05:09.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:05:09.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:05:09.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:05:09.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:05:09.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:05:09.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:05:09.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:05:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:05:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:05:09.780 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:05:39.632 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:05:39.633 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:05:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:05:39.642 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:05:39.675 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:05:39.675 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:05:39.675 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:05:39.675 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:05:39.675 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:05:39.675 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:05:39.675 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:05:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:05:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:05:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:05:49.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:05:49.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:05:49.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:05:49.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:05:49.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:05:49.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:05:49.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:05:49.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:05:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:05:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:05:49.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:06:09.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:06:09.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:06:09.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:06:09.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:06:09.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:06:09.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:06:09.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:06:09.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:06:09.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:06:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:06:09.751 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:06:39.632 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:06:39.633 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:06:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:06:39.640 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:06:39.672 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:06:39.672 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:06:39.672 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:06:39.672 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:06:39.672 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:06:39.672 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:06:39.672 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:06:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:06:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:06:39.674 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:06:49.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:06:49.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:06:49.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:06:49.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:06:49.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:06:49.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:06:49.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:06:49.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:06:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:06:49.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:06:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:07:09.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:07:09.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:07:09.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:07:09.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:07:09.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:07:09.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:07:09.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:07:09.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:07:09.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:07:09.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:07:09.752 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:07:39.633 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:07:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:07:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:07:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:07:39.678 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:07:39.678 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:07:39.678 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:07:39.678 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:07:39.678 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:07:39.678 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:07:39.678 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:07:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:07:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:07:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:07:49.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:07:49.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:07:49.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:07:49.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:07:49.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:07:49.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:07:49.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:07:49.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:07:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:07:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:07:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:08:09.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:08:09.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:08:09.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:08:09.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:08:09.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:08:09.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:08:09.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:08:09.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:08:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:08:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:08:09.753 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:08:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:08:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:08:39.645 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:08:39.646 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:08:39.681 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:08:39.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:08:39.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:08:39.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:08:39.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:08:39.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:08:39.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:08:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:08:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:08:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:08:49.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:08:49.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:08:49.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:08:49.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:08:49.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:08:49.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:08:49.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:08:49.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:08:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:08:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:08:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:09:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:09:09.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:09:09.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:09:09.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:09:09.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:09:09.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:09:09.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:09:09.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:09:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:09:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:09:09.757 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:09:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:09:39.640 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:09:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:09:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:09:39.681 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:09:39.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:09:39.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:09:39.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:09:39.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:09:39.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:09:39.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:09:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:09:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:09:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:09:49.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:09:49.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:09:49.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:09:49.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:09:49.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:09:49.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:09:49.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:09:49.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:09:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:09:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:09:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:10:09.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:10:09.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:10:09.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:10:09.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:10:09.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:10:09.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:10:09.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:10:09.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:10:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:10:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:10:09.756 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:10:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:10:39.638 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:10:39.651 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:10:39.652 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:10:39.684 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:10:39.684 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:10:39.684 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:10:39.684 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:10:39.684 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:10:39.684 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:10:39.684 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:10:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:10:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:10:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:10:49.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:10:49.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:10:49.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:10:49.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:10:49.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:10:49.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:10:49.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:10:49.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:10:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:10:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:10:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:11:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:11:09.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:11:09.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:11:09.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:11:09.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:11:09.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:11:09.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:11:09.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:11:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:11:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:11:09.759 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:11:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:11:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:11:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:11:39.655 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:11:39.691 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:11:39.691 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:11:39.691 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:11:39.691 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:11:39.691 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:11:39.691 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:11:39.691 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:11:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:11:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:11:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:11:49.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:11:49.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:11:49.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:11:49.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:11:49.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:11:49.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:11:49.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:11:49.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:11:49.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:11:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:11:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:12:09.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:12:09.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:12:09.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:12:09.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:12:09.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:12:09.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:12:09.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:12:09.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:12:09.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:12:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:12:09.775 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:12:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:12:39.637 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:12:39.645 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:12:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:12:39.680 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:12:39.680 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:12:39.680 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:12:39.680 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:12:39.680 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:12:39.680 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:12:39.680 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:12:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:12:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:12:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:12:49.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:12:49.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:12:49.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:12:49.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:12:49.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:12:49.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:12:49.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:12:49.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:12:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:12:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:12:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:13:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:13:09.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:13:09.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:13:09.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:13:09.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:13:09.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:13:09.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:13:09.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:13:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:13:09.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:13:09.758 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:13:39.639 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:13:39.641 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:13:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:13:39.648 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:13:39.681 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:13:39.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:13:39.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:13:39.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:13:39.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:13:39.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:13:39.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:13:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:13:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:13:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:13:49.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:13:49.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:13:49.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:13:49.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:13:49.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:13:49.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:13:49.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:13:49.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:13:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:13:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:13:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:14:09.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:14:09.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:14:09.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:14:09.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:14:09.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:14:09.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:14:09.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:14:09.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:14:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:14:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:14:09.769 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:14:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:14:39.641 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:14:39.649 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:14:39.650 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:14:39.696 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:14:39.696 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:14:39.696 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:14:39.696 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:14:39.696 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:14:39.696 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:14:39.696 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:14:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:14:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.048 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:14:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:14:49.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:14:49.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:14:49.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:14:49.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:14:49.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:14:49.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:14:49.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:14:49.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:14:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:14:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:14:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:15:09.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:15:09.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:15:09.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:15:09.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:15:09.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:15:09.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:15:09.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:15:09.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:15:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:15:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:15:09.777 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:15:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:15:39.639 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:15:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:15:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:15:39.685 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:15:39.685 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:15:39.685 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:15:39.685 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:15:39.685 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:15:39.685 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:15:39.685 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:15:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:15:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:15:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:15:49.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:15:49.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:15:49.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:15:49.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:15:49.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:15:49.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:15:49.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:15:49.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:15:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:15:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:15:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:16:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:16:09.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:16:09.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:16:09.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:16:09.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:16:09.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:16:09.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:16:09.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:16:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:16:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:16:09.763 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:16:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:16:39.636 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:16:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:16:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:16:39.678 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:16:39.678 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:16:39.678 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:16:39.678 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:16:39.678 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:16:39.678 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:16:39.678 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:16:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:16:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:16:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:16:49.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:16:49.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:16:49.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:16:49.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:16:49.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:16:49.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:16:49.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:16:49.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:16:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:16:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:16:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:17:09.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:17:09.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:17:09.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:17:09.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:17:09.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:17:09.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:17:09.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:17:09.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:17:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:17:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:17:09.754 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:17:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:17:39.637 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:17:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:17:39.644 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:17:39.676 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:17:39.676 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:17:39.676 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:17:39.676 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:17:39.676 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:17:39.676 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:17:39.676 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:17:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:17:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:17:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:17:49.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:17:49.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:17:49.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:17:49.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:17:49.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:17:49.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:17:49.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:17:49.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:17:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:17:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:17:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:18:09.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:18:09.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:18:09.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:18:09.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:18:09.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:18:09.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:18:09.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:18:09.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:18:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:18:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:18:09.766 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:18:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:18:39.639 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:18:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:18:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:18:39.679 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:18:39.679 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:18:39.679 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:18:39.679 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:18:39.679 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:18:39.679 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:18:39.679 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:18:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:18:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:18:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:18:49.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:18:49.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:18:49.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:18:49.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:18:49.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:18:49.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:18:49.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:18:49.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:18:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:18:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:18:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:19:09.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:19:09.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:19:09.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:19:09.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:19:09.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:19:09.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:19:09.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:19:09.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:19:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:19:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:19:09.756 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:19:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:19:39.638 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:19:39.645 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:19:39.645 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:19:39.678 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:19:39.678 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:19:39.678 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:19:39.678 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:19:39.678 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:19:39.678 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:19:39.678 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:19:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:19:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:19:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:19:49.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:19:49.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:19:49.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:19:49.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:19:49.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:19:49.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:19:49.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:19:49.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:19:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:19:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:19:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:20:09.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:20:09.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:20:09.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:20:09.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:20:09.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:20:09.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:20:09.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:20:09.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:20:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:20:09.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:20:09.753 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:20:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:20:39.636 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:21:39.634 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:21:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:21:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:21:39.643 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:21:39.676 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:21:39.676 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:21:39.676 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:21:39.676 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:21:39.676 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:21:39.676 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:21:39.676 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:21:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:21:39.678 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:21:39.679 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:21:49.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:21:49.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:21:49.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:21:49.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:21:49.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:21:49.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:21:49.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:21:49.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:21:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:21:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:21:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:22:09.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:22:09.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:22:09.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:22:09.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:22:09.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:22:09.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:22:09.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:22:09.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:22:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:22:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:22:09.754 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:22:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:22:39.637 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:22:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:22:39.646 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:22:39.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:22:39.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:22:39.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:22:39.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:22:39.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:22:39.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:22:39.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:22:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:22:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:22:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:22:49.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:22:49.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:22:49.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:22:49.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:22:49.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:22:49.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:22:49.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:22:49.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:22:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:22:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:22:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:23:09.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:23:09.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:23:09.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:23:09.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:23:09.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:23:09.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:23:09.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:23:09.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:23:09.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:23:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:23:09.765 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:23:39.636 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:23:39.637 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:23:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:23:39.647 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:23:39.681 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:23:39.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:23:39.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:23:39.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:23:39.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:23:39.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:23:39.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:23:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:23:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:23:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:23:49.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:23:49.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:23:49.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:23:49.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:23:49.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:23:49.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:23:49.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:23:49.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:23:49.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:23:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.068 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:23:49.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:24:09.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:24:09.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:24:09.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:24:09.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:24:09.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:24:09.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:24:09.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:24:09.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:24:09.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:24:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:24:09.761 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:24:39.632 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:24:39.633 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:24:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:24:39.643 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:24:39.677 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:24:39.677 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:24:39.677 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:24:39.677 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:24:39.677 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:24:39.677 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:24:39.677 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:24:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:24:39.680 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:24:39.681 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:24:49.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:24:49.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:24:49.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:24:49.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:24:49.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:24:49.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:24:49.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:24:49.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:24:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:24:49.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:24:49.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:25:09.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:25:09.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:25:09.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:25:09.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:25:09.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:25:09.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:25:09.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:25:09.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:25:09.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:25:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:25:09.755 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:25:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:25:39.638 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:25:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:25:39.646 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:25:39.683 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:25:39.683 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:25:39.683 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:25:39.683 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:25:39.683 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:25:39.683 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:25:39.683 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:25:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:25:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:25:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:25:49.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:25:49.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:25:49.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:25:49.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:25:49.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:25:49.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:25:49.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:25:49.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:25:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:25:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:25:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:26:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:26:09.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:26:09.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:26:09.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:26:09.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:26:09.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:26:09.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:26:09.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:26:09.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:26:09.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:26:09.759 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:26:39.640 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:26:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:26:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:26:39.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:26:39.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:26:39.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:26:39.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:26:39.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:26:39.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:26:39.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:26:39.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:26:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:26:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:26:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:26:49.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:26:49.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:26:49.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:26:49.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:26:49.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:26:49.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:26:49.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:26:49.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:26:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:26:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:26:49.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:27:09.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:27:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:27:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:27:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:27:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:27:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:27:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:27:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:27:09.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:27:09.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:27:09.772 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:27:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:27:39.645 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:27:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:27:39.655 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:27:39.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:27:39.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:27:39.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:27:39.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:27:39.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:27:39.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:27:39.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:27:39.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:27:39.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:27:39.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:27:49.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:27:49.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:27:49.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:27:49.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:27:49.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:27:49.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:27:49.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:27:49.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:27:49.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:27:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:27:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:28:09.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:28:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:28:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:28:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:28:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:28:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:28:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:28:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:28:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:28:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:28:09.770 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:28:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:28:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:28:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:28:39.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:28:39.684 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:28:39.684 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:28:39.684 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:28:39.684 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:28:39.684 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:28:39.684 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:28:39.684 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:28:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:28:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:28:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:28:49.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:28:49.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:28:49.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:28:49.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:28:49.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:28:49.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:28:49.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:28:49.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:28:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:28:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:28:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:29:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:29:09.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:29:09.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:29:09.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:29:09.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:29:09.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:29:09.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:29:09.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:29:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:29:09.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:29:09.762 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:29:39.643 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:29:39.644 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:29:39.651 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:29:39.652 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:29:39.688 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:29:39.688 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:29:39.688 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:29:39.688 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:29:39.688 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:29:39.688 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:29:39.688 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:29:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:29:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:29:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:29:49.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:29:49.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:29:49.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:29:49.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:29:49.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:29:49.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:29:49.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:29:49.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:29:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:29:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:29:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:30:09.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:30:09.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:30:09.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:30:09.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:30:09.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:30:09.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:30:09.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:30:09.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:30:09.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:30:09.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:30:09.770 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:30:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:30:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:30:39.648 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:30:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:30:39.683 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:30:39.683 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:30:39.683 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:30:39.683 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:30:39.683 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:30:39.683 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:30:39.683 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:30:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:30:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:30:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:30:49.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:30:49.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:30:49.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:30:49.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:30:49.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:30:49.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:30:49.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:30:49.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:30:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:30:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:30:49.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:31:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:31:09.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:31:09.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:31:09.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:31:09.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:31:09.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:31:09.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:31:09.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:31:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:31:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:31:09.756 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:31:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:31:39.639 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:31:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:31:39.646 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:31:39.681 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:31:39.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:31:39.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:31:39.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:31:39.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:31:39.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:31:39.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:31:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:31:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:31:39.684 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:31:49.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:31:49.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:31:49.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:31:49.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:31:49.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:31:49.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:31:49.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:31:49.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:31:49.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:31:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:31:49.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:32:09.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:32:09.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:32:09.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:32:09.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:32:09.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:32:09.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:32:09.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:32:09.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:32:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:32:09.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:32:09.757 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:32:39.638 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:32:39.639 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:32:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:32:39.648 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:32:39.683 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:32:39.683 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:32:39.683 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:32:39.683 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:32:39.683 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:32:39.683 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:32:39.683 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:32:39.685 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:32:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:32:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:32:49.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:32:49.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:32:49.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:32:49.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:32:49.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:32:49.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:32:49.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:32:49.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:32:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:32:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:32:49.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:33:09.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:33:09.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:33:09.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:33:09.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:33:09.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:33:09.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:33:09.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:33:09.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:33:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:33:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:33:09.763 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:33:39.635 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:33:39.635 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:33:39.649 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:33:39.649 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:33:39.681 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:33:39.681 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:33:39.681 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:33:39.681 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:33:39.681 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:33:39.681 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:33:39.681 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:33:39.682 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:33:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:33:39.683 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:33:49.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:33:49.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:33:49.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:33:49.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:33:49.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:33:49.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:33:49.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:33:49.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:33:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:33:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:33:49.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:34:09.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:34:09.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:34:09.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:34:09.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:34:09.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:34:09.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:34:09.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:34:09.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:34:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:34:09.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:34:09.756 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:34:39.637 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:34:39.638 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:34:39.651 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:34:39.652 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:34:39.684 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:34:39.684 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:34:39.684 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:34:39.684 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:34:39.684 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:34:39.684 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:34:39.684 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:34:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:34:39.686 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:34:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:34:49.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:34:49.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:34:49.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:34:49.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:34:49.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:34:49.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:34:49.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:34:49.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:34:49.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:34:49.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:34:49.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:35:09.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:35:09.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:35:09.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:35:09.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:35:09.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:35:09.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:35:09.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:35:09.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:35:09.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:35:09.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:35:09.760 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:35:39.642 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:35:39.643 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:35:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:35:39.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:35:39.685 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:35:39.685 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:35:39.685 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:35:39.685 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:35:39.685 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:35:39.685 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:35:39.685 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:35:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:35:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:35:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:35:49.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:35:49.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:35:49.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:35:49.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:35:49.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:35:49.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:35:49.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:35:49.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:35:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:35:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:35:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:36:09.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:36:09.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:36:09.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:36:09.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:36:09.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:36:09.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:36:09.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:36:09.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:36:09.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:36:09.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:36:09.770 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:36:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:36:39.643 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:36:39.650 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:36:39.651 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:36:39.686 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:36:39.686 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:36:39.686 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:36:39.686 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:36:39.686 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:36:39.686 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:36:39.686 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:36:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:36:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:36:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:36:49.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:36:49.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:36:49.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:36:49.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:36:49.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:36:49.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:36:49.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:36:49.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:36:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:36:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:36:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:37:09.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:37:09.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:37:09.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:37:09.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:37:09.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:37:09.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:37:09.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:37:09.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:37:09.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:37:09.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:37:09.769 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:37:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:37:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:37:39.651 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:37:39.652 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:37:39.685 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:37:39.685 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:37:39.685 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:37:39.685 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:37:39.685 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:37:39.685 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:37:39.685 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:37:39.687 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:37:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:37:39.688 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:37:49.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:37:49.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:37:49.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:37:49.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:37:49.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:37:49.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:37:49.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:37:49.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:37:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:37:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:37:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:38:09.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:38:09.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:38:09.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:38:09.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:38:09.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:38:09.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:38:09.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:38:09.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:38:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:38:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:38:09.763 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:38:39.645 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:38:39.645 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:38:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:38:39.657 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:38:39.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:38:39.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:38:39.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:38:39.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:38:39.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:38:39.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:38:39.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:38:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:38:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:38:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:38:49.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:38:49.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:38:49.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:38:49.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:38:49.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:38:49.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:38:49.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:38:49.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:38:49.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:38:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:38:49.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:39:09.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:39:09.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:39:09.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:39:09.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:39:09.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:39:09.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:39:09.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:39:09.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:39:09.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:39:09.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:39:09.770 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:39:39.641 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:39:39.642 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:39:39.652 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:39:39.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:39:39.688 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:39:39.688 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:39:39.688 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:39:39.688 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:39:39.688 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:39:39.688 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:39:39.688 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:39:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:39:39.690 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:39:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:39:49.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:39:49.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:39:49.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:39:49.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:39:49.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:39:49.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:39:49.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:39:49.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:39:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:39:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:39:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:40:09.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:40:09.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:40:09.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:40:09.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:40:09.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:40:09.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:40:09.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:40:09.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:40:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:40:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:40:09.764 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:40:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:40:39.647 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:40:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:40:39.654 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:40:39.686 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:40:39.686 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:40:39.686 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:40:39.686 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:40:39.686 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:40:39.686 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:40:39.686 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:40:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:40:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:40:39.689 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:40:49.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:40:49.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:40:49.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:40:49.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:40:49.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:40:49.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:40:49.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:40:49.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:40:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:40:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:40:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:41:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:41:09.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:41:09.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:41:09.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:41:09.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:41:09.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:41:09.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:41:09.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:41:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:41:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:41:09.762 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:41:39.644 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:41:39.645 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:41:39.652 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:41:39.653 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:41:39.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:41:39.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:41:39.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:41:39.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:41:39.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:41:39.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:41:39.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:41:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:41:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:41:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:41:49.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:41:49.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:41:49.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:41:49.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:41:49.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:41:49.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:41:49.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:41:49.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:41:49.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:41:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:41:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:42:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:42:09.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:42:09.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:42:09.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:42:09.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:42:09.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:42:09.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:42:09.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:42:09.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:42:09.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:42:09.764 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:42:39.646 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:42:39.647 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:42:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:42:39.655 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:42:39.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:42:39.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:42:39.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:42:39.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:42:39.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:42:39.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:42:39.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:42:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:42:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:42:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:42:49.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:42:49.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:42:49.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:42:49.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:42:49.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:42:49.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:42:49.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:42:49.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:42:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:42:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:42:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:43:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:43:09.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:43:09.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:43:09.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:43:09.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:43:09.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:43:09.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:43:09.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:43:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:43:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:43:09.766 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:43:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:43:39.648 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:43:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:43:39.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:43:39.691 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:43:39.691 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:43:39.691 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:43:39.691 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:43:39.691 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:43:39.691 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:43:39.691 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:43:39.693 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:43:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:43:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:43:49.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:43:49.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:43:49.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:43:49.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:43:49.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:43:49.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:43:49.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:43:49.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:43:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:43:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:43:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:44:09.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:44:09.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:44:09.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:44:09.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:44:09.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:44:09.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:44:09.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:44:09.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:44:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:44:09.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:44:09.770 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:44:39.652 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:44:39.653 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:44:39.660 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:44:39.660 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:44:39.695 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:44:39.695 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:44:39.695 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:44:39.695 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:44:39.695 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:44:39.695 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:44:39.695 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:44:39.697 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:44:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:44:39.698 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:44:49.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:44:49.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:44:49.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:44:49.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:44:49.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:44:49.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:44:49.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:44:49.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:44:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:44:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:44:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:45:09.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:45:09.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:45:09.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:45:09.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:45:09.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:45:09.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:45:09.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:45:09.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:45:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:45:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:45:09.776 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:45:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:45:39.648 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:45:39.656 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:45:39.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:45:39.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:45:39.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:45:39.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:45:39.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:45:39.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:45:39.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:45:39.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:45:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:45:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:45:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:45:49.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:45:49.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:45:49.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:45:49.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:45:49.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:45:49.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:45:49.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:45:49.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:45:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:45:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:45:49.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:46:09.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:46:09.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:46:09.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:46:09.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:46:09.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:46:09.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:46:09.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:46:09.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:46:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:46:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:46:09.766 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:46:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:46:39.649 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:46:39.656 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:46:39.656 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:46:39.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:46:39.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:46:39.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:46:39.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:46:39.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:46:39.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:46:39.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:46:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:46:39.694 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:46:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:46:49.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:46:49.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:46:49.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:46:49.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:46:49.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:46:49.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:46:49.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:46:49.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:46:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:46:49.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:46:49.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:47:09.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:47:09.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:47:09.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:47:09.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:47:09.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:47:09.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:47:09.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:47:09.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:47:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:47:09.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:47:09.766 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:47:39.647 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:47:39.648 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:47:39.655 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:47:39.655 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:47:39.689 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:47:39.689 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:47:39.689 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:47:39.689 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:47:39.689 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:47:39.689 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:47:39.689 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:47:39.691 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:47:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:47:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:47:49.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:47:49.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:47:49.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:47:49.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:47:49.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:47:49.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:47:49.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:47:49.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:47:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:47:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:47:49.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:48:09.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:48:09.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:48:09.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:48:09.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:48:09.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:48:09.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:48:09.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:48:09.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:48:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:48:09.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:48:09.767 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:48:39.648 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:48:39.649 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:48:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:48:39.657 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:48:39.692 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:48:39.692 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:48:39.692 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:48:39.692 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:48:39.692 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:48:39.692 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:48:39.692 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:48:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:48:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:48:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:48:49.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:48:49.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:48:49.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:48:49.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:48:49.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:48:49.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:48:49.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:48:49.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:48:49.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:48:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:48:49.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:49:09.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:49:09.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:49:09.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:49:09.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:49:09.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:49:09.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:49:09.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:49:09.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:49:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:49:09.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:49:09.774 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:49:39.645 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:49:39.646 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:49:39.659 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:49:39.659 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:49:39.693 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:49:39.693 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:49:39.693 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:49:39.693 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:49:39.693 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:49:39.693 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:49:39.693 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:49:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:49:39.695 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:49:39.696 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:49:49.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:49:49.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:49:49.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:49:49.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:49:49.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:49:49.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:49:49.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:49:49.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:49:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:49:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:49:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:50:09.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:50:09.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:50:09.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:50:09.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:50:09.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:50:09.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:50:09.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:50:09.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:50:09.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:50:09.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:50:09.778 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:50:39.649 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:50:39.650 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:50:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:50:39.658 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:50:39.690 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:50:39.690 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:50:39.690 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:50:39.690 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:50:39.690 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:50:39.690 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:50:39.690 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:50:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:50:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:50:39.692 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:50:49.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:50:49.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:50:49.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:50:49.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:50:49.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:50:49.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:50:49.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:50:49.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:50:49.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:50:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:50:49.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:51:09.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:51:09.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:51:09.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:51:09.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:51:09.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:51:09.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:51:09.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:51:09.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:51:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:51:09.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:51:09.763 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:51:39.654 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:51:39.655 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:51:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:51:39.662 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:51:39.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:51:39.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:51:39.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:51:39.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:51:39.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:51:39.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:51:39.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:51:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:51:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:51:39.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:51:49.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:51:49.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:51:49.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:51:49.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:51:49.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:51:49.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:51:49.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:51:49.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:51:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:51:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:51:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:52:09.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:52:09.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:52:09.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:52:09.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:52:09.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:52:09.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:52:09.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:52:09.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:52:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:52:09.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:52:09.776 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:52:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:52:39.658 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:52:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:52:39.667 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:52:39.697 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:52:39.697 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:52:39.697 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:52:39.697 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:52:39.697 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:52:39.697 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:52:39.697 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:52:39.699 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:52:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:52:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:52:49.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:52:49.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:52:49.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:52:49.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:52:49.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:52:49.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:52:49.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:52:49.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:52:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:52:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:52:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:53:09.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:53:09.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:53:09.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:53:09.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:53:09.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:53:09.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:53:09.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:53:09.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:53:09.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:53:09.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:53:09.774 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:53:39.656 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:53:39.657 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:53:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:53:39.667 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:53:39.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:53:39.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:53:39.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:53:39.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:53:39.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:53:39.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:53:39.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:53:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:53:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:53:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:53:49.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:53:49.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:53:49.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:53:49.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:53:49.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:53:49.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:53:49.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:53:49.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:53:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:53:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:53:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:54:09.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:54:09.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:54:09.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:54:09.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:54:09.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:54:09.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:54:09.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:54:09.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:54:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:54:09.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:54:09.779 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:54:39.661 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:54:39.662 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:54:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:54:39.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:54:39.705 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:54:39.705 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:54:39.705 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:54:39.705 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:54:39.705 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:54:39.705 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:54:39.705 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:54:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:54:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:54:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:54:49.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:54:49.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:54:49.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:54:49.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:54:49.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:54:49.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:54:49.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:54:49.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:54:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:54:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:54:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:55:09.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:55:09.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:55:09.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:55:09.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:55:09.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:55:09.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:55:09.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:55:09.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:55:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:55:09.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:55:09.779 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:55:39.660 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:55:39.661 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:55:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:55:39.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:55:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:55:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:55:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:55:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:55:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:55:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:55:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:55:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:55:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:55:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:55:49.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:55:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:55:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:55:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:55:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:55:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:55:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:55:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:55:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:55:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:55:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:56:09.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:56:09.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:56:09.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:56:09.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:56:09.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:56:09.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:56:09.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:56:09.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:56:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:56:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:56:09.787 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:56:39.658 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:56:39.660 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:56:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:56:39.667 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:56:39.702 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:56:39.702 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:56:39.702 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:56:39.702 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:56:39.702 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:56:39.702 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:56:39.702 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:56:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:56:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:56:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:56:49.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:56:49.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:56:49.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:56:49.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:56:49.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:56:49.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:56:49.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:56:49.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:56:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:56:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:56:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:57:09.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:57:09.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:57:09.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:57:09.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:57:09.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:57:09.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:57:09.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:57:09.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:57:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:57:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:57:09.781 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:57:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:57:39.664 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:57:39.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:57:39.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:57:39.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:57:39.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:57:39.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:57:39.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:57:39.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:57:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:57:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:57:39.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:57:49.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:57:49.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:57:49.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:57:49.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:57:49.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:57:49.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:57:49.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:57:49.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:57:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:57:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:57:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:58:09.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:58:09.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:58:09.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:58:09.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:58:09.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:58:09.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:58:09.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:58:09.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:58:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:58:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:58:09.776 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:58:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:58:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:58:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:58:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:58:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:58:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:58:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:58:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:58:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:58:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:58:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:58:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:58:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:58:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:58:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:58:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:58:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:58:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:58:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:58:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:58:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:58:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:58:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:58:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.091 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:58:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:59:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:59:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:59:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:59:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:59:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:59:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:59:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:59:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:59:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:59:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.132 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 18:59:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 18:59:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:59:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 18:59:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 18:59:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:59:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:59:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:59:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:59:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:59:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:59:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:59:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:59:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:59:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:59:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 18:59:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 18:59:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 18:59:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 18:59:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 18:59:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 18:59:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 18:59:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 18:59:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 18:59:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 18:59:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 18:59:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:00:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:00:09.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:00:09.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:00:09.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:00:09.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:00:09.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:00:09.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:00:09.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:00:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:00:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:00:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:00:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:00:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:00:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:00:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:00:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:00:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:00:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:00:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:00:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:00:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:00:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:00:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:00:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:00:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:00:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:00:49.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:00:49.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:00:49.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:00:49.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:00:49.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:00:49.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:00:49.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:00:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:00:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:00:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:01:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:01:09.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:01:09.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:01:09.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:01:09.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:01:09.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:01:09.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:01:09.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:01:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:01:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:01:09.801 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:01:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:01:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:01:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:01:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:01:39.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:01:39.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:01:39.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:01:39.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:01:39.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:01:39.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:01:39.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:01:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:01:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:01:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:01:49.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:01:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:01:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:01:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:01:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:01:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:01:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:01:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:01:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:01:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:01:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:02:09.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:02:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:02:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:02:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:02:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:02:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:02:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:02:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:02:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:02:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:02:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:02:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:02:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:02:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:02:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:02:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:02:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:02:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:02:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:02:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:02:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:02:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:02:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:02:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:02:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:02:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:02:49.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:02:49.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:02:49.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:02:49.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:02:49.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:02:49.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:02:49.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:02:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:02:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:02:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:03:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:03:09.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:03:09.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:03:09.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:03:09.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:03:09.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:03:09.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:03:09.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:03:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:03:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:03:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:03:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:03:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:03:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:03:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:03:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:03:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:03:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:03:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:03:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:03:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:03:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:03:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:03:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:03:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:03:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:03:49.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:03:49.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:03:49.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:03:49.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:03:49.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:03:49.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:03:49.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:03:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:03:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.093 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:03:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:04:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:04:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:04:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:04:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:04:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:04:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:04:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:04:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:04:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:04:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:04:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:04:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:04:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:04:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:04:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:04:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:04:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:04:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:04:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:04:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:04:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:04:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:04:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:04:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:04:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:04:49.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:04:49.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:04:49.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:04:49.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:04:49.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:04:49.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:04:49.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:04:49.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:04:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:04:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:04:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:05:09.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:05:09.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:05:09.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:05:09.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:05:09.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:05:09.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:05:09.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:05:09.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:05:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:05:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:05:09.792 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:05:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:05:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:05:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:05:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:05:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:05:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:05:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:05:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:05:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:05:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:05:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:05:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:05:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:05:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:05:49.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:05:49.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:05:49.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:05:49.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:05:49.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:05:49.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:05:49.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:05:49.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:05:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:05:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:05:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:06:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:06:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:06:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:06:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:06:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:06:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:06:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:06:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:06:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:06:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:06:09.805 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:06:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:06:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:06:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:06:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:06:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:06:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:06:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:06:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:06:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:06:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:06:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:06:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:06:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:06:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:06:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:06:49.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:06:49.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:06:49.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:06:49.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:06:49.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:06:49.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:06:49.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:06:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:06:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:06:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:07:09.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:07:09.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:07:09.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:07:09.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:07:09.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:07:09.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:07:09.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:07:09.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:07:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:07:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:07:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:07:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:07:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:07:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:07:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:07:39.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:07:39.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:07:39.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:07:39.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:07:39.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:07:39.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:07:39.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:07:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:07:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:07:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:07:49.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:07:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:07:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:07:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:07:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:07:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:07:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:07:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:07:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:07:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:07:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:08:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:08:09.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:08:09.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:08:09.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:08:09.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:08:09.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:08:09.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:08:09.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:08:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:08:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:08:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:08:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:08:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:08:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:08:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:08:39.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:08:39.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:08:39.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:08:39.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:08:39.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:08:39.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:08:39.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:08:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:08:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:08:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:08:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:08:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:08:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:08:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:08:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:08:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:08:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:08:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:08:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:08:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:08:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:09:09.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:09:09.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:09:09.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:09:09.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:09:09.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:09:09.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:09:09.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:09:09.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:09:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:09:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:09:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:09:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:09:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:09:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:09:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:09:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:09:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:09:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:09:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:09:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:09:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:09:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:09:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:09:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:09:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:09:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:09:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:09:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:09:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:09:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:09:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:09:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:09:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:09:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:09:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:09:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:10:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:10:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:10:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:10:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:10:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:10:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:10:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:10:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:10:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:10:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:10:09.794 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:10:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:10:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:10:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:10:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:10:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:10:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:10:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:10:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:10:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:10:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:10:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:10:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:10:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:10:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:10:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:10:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:10:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:10:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:10:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:10:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:10:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:10:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:10:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:10:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:10:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:11:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:11:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:11:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:11:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:11:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:11:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:11:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:11:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:11:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:11:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:11:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:11:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:11:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:11:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:11:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:11:39.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:11:39.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:11:39.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:11:39.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:11:39.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:11:39.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:11:39.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:11:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:11:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:11:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:11:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:11:49.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:11:49.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:11:49.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:11:49.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:11:49.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:11:49.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:11:49.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:11:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:11:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:11:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:12:09.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:12:09.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:12:09.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:12:09.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:12:09.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:12:09.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:12:09.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:12:09.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:12:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:12:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:12:09.798 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:12:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:12:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:12:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:12:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:12:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:12:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:12:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:12:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:12:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:12:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:12:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:12:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:12:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:12:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:12:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:12:49.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:12:49.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:12:49.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:12:49.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:12:49.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:12:49.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:12:49.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:12:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:12:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:12:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:13:09.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:13:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:13:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:13:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:13:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:13:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:13:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:13:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:13:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:13:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:13:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:13:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:13:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:13:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:13:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:13:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:13:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:13:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:13:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:13:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:13:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:13:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:13:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:13:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:13:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:13:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:13:49.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:13:49.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:13:49.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:13:49.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:13:49.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:13:49.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:13:49.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:13:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:13:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:13:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:14:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:14:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:14:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:14:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:14:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:14:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:14:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:14:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:14:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:14:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:14:09.805 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:14:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:14:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:14:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:14:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:14:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:14:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:14:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:14:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:14:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:14:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:14:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:14:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:14:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:14:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:14:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:14:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:14:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:14:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:14:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:14:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:14:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:14:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:14:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:14:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:14:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:15:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:15:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:15:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:15:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:15:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:15:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:15:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:15:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:15:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:15:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:15:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:15:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:15:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:15:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:15:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:15:39.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:15:39.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:15:39.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:15:39.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:15:39.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:15:39.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:15:39.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:15:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:15:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:15:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:15:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:15:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:15:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:15:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:15:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:15:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:15:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:15:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:15:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:15:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:15:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:16:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:16:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:16:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:16:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:16:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:16:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:16:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:16:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:16:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:16:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:16:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:16:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:16:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:16:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:16:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:16:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:16:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:16:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:16:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:16:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:16:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:16:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:16:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:16:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:16:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:16:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:16:49.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:16:49.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:16:49.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:16:49.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:16:49.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:16:49.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:16:49.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:16:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:16:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:16:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:17:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:17:09.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:17:09.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:17:09.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:17:09.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:17:09.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:17:09.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:17:09.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:17:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:17:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:17:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:17:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:17:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:17:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:17:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:17:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:17:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:17:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:17:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:17:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:17:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:17:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:17:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:17:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:17:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:17:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:17:49.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:17:49.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:17:49.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:17:49.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:17:49.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:17:49.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:17:49.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:17:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:17:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:17:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:18:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:18:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:18:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:18:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:18:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:18:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:18:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:18:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:18:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:18:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:18:09.792 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:18:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:18:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:18:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:18:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:18:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:18:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:18:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:18:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:18:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:18:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:18:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:18:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:18:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:18:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:18:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:18:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:18:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:18:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:18:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:18:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:18:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:18:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:18:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:18:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:18:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:19:09.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:19:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:19:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:19:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:19:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:19:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:19:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:19:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:19:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:19:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:19:09.808 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:19:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:19:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:19:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:19:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:19:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:19:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:19:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:19:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:19:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:19:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:19:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:19:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:19:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:19:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:19:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:19:49.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:19:49.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:19:49.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:19:49.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:19:49.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:19:49.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:19:49.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:19:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:19:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:19:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:20:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:20:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:20:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:20:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:20:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:20:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:20:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:20:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:20:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:20:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:20:09.794 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:20:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:20:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:20:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:20:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:20:39.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:20:39.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:20:39.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:20:39.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:20:39.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:20:39.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:20:39.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:20:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:20:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:20:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:20:49.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:20:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:20:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:20:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:20:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:20:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:20:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:20:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:20:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:20:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:20:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:21:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:21:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:21:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:21:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:21:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:21:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:21:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:21:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:21:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:21:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:21:09.795 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:21:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:21:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:21:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:21:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:21:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:21:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:21:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:21:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:21:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:21:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:21:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:21:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:21:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:21:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:21:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:21:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:21:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:21:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:21:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:21:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:21:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:21:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:21:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:21:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:21:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:22:09.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:22:09.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:22:09.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:22:09.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:22:09.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:22:09.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:22:09.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:22:09.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:22:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:22:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:22:09.792 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:22:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:22:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:22:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:22:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:22:39.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:22:39.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:22:39.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:22:39.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:22:39.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:22:39.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:22:39.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:22:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:22:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:22:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:22:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:22:49.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:22:49.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:22:49.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:22:49.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:22:49.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:22:49.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:22:49.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:22:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:22:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.067 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:22:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:23:09.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:23:09.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:23:09.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:23:09.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:23:09.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:23:09.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:23:09.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:23:09.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:23:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:23:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.104 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:23:09.787 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:23:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:23:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:23:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:23:39.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:23:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:23:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:23:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:23:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:23:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:23:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:23:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:23:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:23:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:23:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:23:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:23:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:23:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:23:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:23:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:23:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:23:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:23:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:23:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:23:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:23:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:24:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:24:09.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:24:09.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:24:09.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:24:09.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:24:09.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:24:09.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:24:09.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:24:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:24:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:24:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:24:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:24:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:24:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:24:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:24:39.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:24:39.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:24:39.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:24:39.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:24:39.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:24:39.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:24:39.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:24:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:24:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:24:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:24:49.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:24:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:24:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:24:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:24:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:24:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:24:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:24:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:24:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:24:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:24:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:25:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:25:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:25:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:25:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:25:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:25:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:25:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:25:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:25:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:25:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:25:09.791 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:25:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:25:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:25:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:25:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:25:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:25:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:25:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:25:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:25:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:25:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:25:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:25:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:25:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:25:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:25:49.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:25:49.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:25:49.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:25:49.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:25:49.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:25:49.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:25:49.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:25:49.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:25:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:25:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:25:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:26:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:26:09.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:26:09.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:26:09.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:26:09.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:26:09.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:26:09.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:26:09.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:26:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:26:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:26:09.789 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:26:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:26:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:26:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:26:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:26:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:26:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:26:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:26:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:26:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:26:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:26:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:26:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:26:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:26:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:26:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:26:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:26:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:26:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:26:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:26:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:26:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:26:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:26:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:26:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:26:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:27:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:27:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:27:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:27:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:27:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:27:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:27:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:27:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:27:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:27:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:27:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:27:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:27:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:27:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:27:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:27:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:27:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:27:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:27:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:27:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:27:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:27:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:27:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:27:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:27:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:27:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:27:49.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:27:49.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:27:49.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:27:49.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:27:49.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:27:49.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:27:49.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:27:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:27:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:27:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:28:09.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:28:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:28:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:28:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:28:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:28:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:28:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:28:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:28:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:28:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:28:09.791 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:28:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:28:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:28:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:28:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:28:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:28:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:28:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:28:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:28:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:28:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:28:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:28:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:28:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:28:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:28:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:28:49.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:28:49.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:28:49.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:28:49.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:28:49.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:28:49.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:28:49.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:28:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:28:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:28:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:29:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:29:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:29:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:29:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:29:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:29:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:29:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:29:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:29:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:29:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:29:09.795 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:29:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:29:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:29:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:29:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:29:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:29:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:29:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:29:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:29:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:29:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:29:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:29:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:29:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:29:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:29:49.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:29:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:29:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:29:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:29:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:29:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:29:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:29:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:29:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:29:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:29:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:30:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:30:09.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:30:09.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:30:09.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:30:09.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:30:09.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:30:09.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:30:09.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:30:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:30:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:30:09.798 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:30:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:30:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:30:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:30:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:30:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:30:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:30:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:30:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:30:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:30:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:30:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:30:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:30:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:30:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:30:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:30:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:30:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:30:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:30:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:30:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:30:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:30:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:30:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:30:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:30:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:31:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:31:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:31:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:31:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:31:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:31:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:31:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:31:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:31:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:31:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:31:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:31:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:31:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:32:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:32:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:32:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:32:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:32:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:32:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:32:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:32:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:32:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:32:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:32:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:32:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:32:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:32:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:32:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:32:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:32:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:32:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:32:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:32:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:32:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:32:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:32:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:32:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:32:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:33:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:33:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:33:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:33:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:33:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:33:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:33:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:33:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:33:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:33:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:33:09.789 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:33:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:33:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:33:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:33:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:33:39.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:33:39.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:33:39.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:33:39.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:33:39.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:33:39.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:33:39.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:33:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:33:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:33:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:33:49.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:33:49.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:33:49.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:33:49.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:33:49.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:33:49.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:33:49.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:33:49.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:33:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:33:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:33:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:34:09.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:34:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:34:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:34:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:34:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:34:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:34:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:34:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:34:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:34:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:34:09.796 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:34:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:34:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:34:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:34:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:34:39.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:34:39.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:34:39.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:34:39.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:34:39.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:34:39.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:34:39.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:34:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:34:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:34:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:34:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:34:49.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:34:49.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:34:49.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:34:49.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:34:49.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:34:49.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:34:49.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:34:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:34:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:34:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:35:09.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:35:09.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:35:09.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:35:09.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:35:09.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:35:09.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:35:09.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:35:09.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:35:09.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:35:09.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:35:09.783 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:35:39.665 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:35:39.666 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:35:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:35:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:35:39.706 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:35:39.706 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:35:39.706 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:35:39.706 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:35:39.706 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:35:39.706 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:35:39.706 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:35:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:35:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:35:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:35:49.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:35:49.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:35:49.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:35:49.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:35:49.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:35:49.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:35:49.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:35:49.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:35:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:35:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:35:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:36:09.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:36:09.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:36:09.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:36:09.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:36:09.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:36:09.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:36:09.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:36:09.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:36:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:36:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:36:09.783 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:36:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:36:39.665 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:36:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:36:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:36:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:36:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:36:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:36:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:36:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:36:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:36:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:36:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:36:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:36:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:36:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:36:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:36:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:36:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:36:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:36:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:36:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:36:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:36:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:36:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:36:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:37:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:37:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:37:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:37:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:37:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:37:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:37:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:37:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:37:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:37:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.133 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:37:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:37:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:37:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:37:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:37:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:37:39.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:37:39.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:37:39.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:37:39.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:37:39.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:37:39.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:37:39.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:37:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:37:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:37:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:37:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:37:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:37:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:37:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:37:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:37:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:37:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:37:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:37:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:37:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:37:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:38:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:38:09.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:38:09.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:38:09.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:38:09.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:38:09.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:38:09.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:38:09.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:38:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:38:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:38:09.788 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:38:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:38:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:38:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:38:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:38:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:38:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:38:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:38:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:38:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:38:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:38:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:38:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:38:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:38:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:38:49.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:38:49.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:38:49.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:38:49.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:38:49.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:38:49.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:38:49.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:38:49.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:38:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:38:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:38:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:39:09.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:39:09.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:39:09.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:39:09.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:39:09.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:39:09.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:39:09.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:39:09.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:39:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:39:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:39:09.784 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:39:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:39:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:39:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:39:39.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:39:39.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:39:39.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:39:39.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:39:39.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:39:39.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:39:39.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:39:39.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:39:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:39:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:39:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:39:49.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:39:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:39:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:39:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:39:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:39:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:39:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:39:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:39:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:39:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:39:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:40:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:40:09.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:40:09.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:40:09.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:40:09.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:40:09.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:40:09.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:40:09.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:40:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:40:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:40:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:40:39.665 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:40:39.666 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:40:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:40:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:40:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:40:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:40:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:40:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:40:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:40:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:40:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:40:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:40:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:40:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:40:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:40:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:40:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:40:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:40:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:40:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:40:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:40:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:40:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:40:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:40:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:41:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:41:09.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:41:09.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:41:09.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:41:09.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:41:09.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:41:09.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:41:09.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:41:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:41:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:41:09.794 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:41:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:41:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:41:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:41:39.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:41:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:41:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:41:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:41:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:41:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:41:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:41:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:41:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:41:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:41:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:41:49.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:41:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:41:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:41:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:41:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:41:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:41:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:41:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:41:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:41:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:41:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:42:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:42:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:42:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:42:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:42:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:42:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:42:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:42:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:42:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:42:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:42:09.788 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:42:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:42:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:42:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:42:39.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:42:39.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:42:39.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:42:39.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:42:39.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:42:39.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:42:39.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:42:39.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:42:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:42:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:42:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:42:49.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:42:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:42:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:42:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:42:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:42:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:42:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:42:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:42:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:42:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:42:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:43:09.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:43:09.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:43:09.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:43:09.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:43:09.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:43:09.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:43:09.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:43:09.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:43:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:43:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:43:09.788 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:43:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:43:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:43:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:43:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:43:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:43:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:43:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:43:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:43:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:43:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:43:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:43:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:43:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:43:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:43:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:43:49.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:43:49.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:43:49.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:43:49.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:43:49.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:43:49.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:43:49.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:43:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:43:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:43:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:44:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:44:09.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:44:09.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:44:09.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:44:09.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:44:09.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:44:09.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:44:09.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:44:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:44:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:44:09.794 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:44:39.665 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:44:39.666 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:44:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:44:39.674 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:44:39.708 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:44:39.708 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:44:39.708 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:44:39.708 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:44:39.708 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:44:39.708 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:44:39.708 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:44:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:44:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:44:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:44:49.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:44:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:44:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:44:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:44:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:44:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:44:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:44:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:44:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:44:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:44:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:45:09.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:45:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:45:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:45:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:45:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:45:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:45:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:45:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:45:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:45:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:45:09.789 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:45:39.660 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:45:39.662 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:45:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:45:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:45:39.706 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:45:39.706 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:45:39.706 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:45:39.706 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:45:39.706 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:45:39.706 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:45:39.706 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:45:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:45:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:45:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:45:49.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:45:49.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:45:49.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:45:49.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:45:49.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:45:49.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:45:49.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:45:49.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:45:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:45:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:45:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:46:09.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:46:09.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:46:09.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:46:09.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:46:09.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:46:09.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:46:09.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:46:09.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:46:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:46:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:46:09.781 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:46:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:46:39.664 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:46:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:46:39.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:46:39.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:46:39.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:46:39.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:46:39.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:46:39.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:46:39.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:46:39.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:46:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:46:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:46:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:46:49.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:46:49.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:46:49.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:46:49.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:46:49.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:46:49.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:46:49.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:46:49.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:46:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:46:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:46:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:47:09.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:47:09.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:47:09.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:47:09.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:47:09.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:47:09.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:47:09.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:47:09.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:47:09.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:47:09.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:47:09.777 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:47:39.658 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:47:39.659 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:47:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:47:39.667 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:47:39.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:47:39.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:47:39.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:47:39.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:47:39.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:47:39.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:47:39.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:47:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:47:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:47:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:47:49.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:47:49.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:47:49.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:47:49.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:47:49.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:47:49.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:47:49.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:47:49.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:47:49.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:47:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:47:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:48:09.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:48:09.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:48:09.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:48:09.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:48:09.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:48:09.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:48:09.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:48:09.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:48:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:48:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:48:09.781 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:48:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:48:39.663 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:48:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:48:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:48:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:48:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:48:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:48:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:48:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:48:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:48:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:48:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:48:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:48:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:48:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:48:49.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:48:49.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:48:49.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:48:49.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:48:49.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:48:49.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:48:49.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:48:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:48:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:48:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:49:09.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:49:09.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:49:09.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:49:09.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:49:09.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:49:09.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:49:09.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:49:09.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:49:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:49:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:49:09.785 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:49:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:49:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:49:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:49:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:49:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:49:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:49:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:49:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:49:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:49:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:49:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:49:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:49:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:49:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:49:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:49:49.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:49:49.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:49:49.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:49:49.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:49:49.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:49:49.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:49:49.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:49:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:49:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:49:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:50:09.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:50:09.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:50:09.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:50:09.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:50:09.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:50:09.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:50:09.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:50:09.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:50:09.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:50:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:50:09.784 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:50:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:50:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:50:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:50:39.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:50:39.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:50:39.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:50:39.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:50:39.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:50:39.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:50:39.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:50:39.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:50:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:50:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:50:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:50:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:50:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:50:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:50:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:50:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:50:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:50:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:50:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:50:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:50:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:50:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:51:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:51:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:51:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:51:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:51:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:51:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:51:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:51:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:51:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:51:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:51:09.789 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:51:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:51:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:51:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:51:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:51:39.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:51:39.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:51:39.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:51:39.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:51:39.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:51:39.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:51:39.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:51:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:51:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:51:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:51:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:51:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:51:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:51:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:51:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:51:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:51:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:51:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:51:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:51:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:51:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:52:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:52:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:52:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:52:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:52:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:52:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:52:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:52:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:52:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:52:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:52:09.799 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:52:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:52:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:52:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:52:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:52:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:52:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:52:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:52:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:52:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:52:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:52:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:52:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:52:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:52:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:52:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:52:49.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:52:49.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:52:49.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:52:49.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:52:49.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:52:49.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:52:49.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:52:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:52:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:52:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:53:09.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:53:09.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:53:09.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:53:09.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:53:09.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:53:09.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:53:09.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:53:09.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:53:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:53:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:53:09.787 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:53:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:53:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:53:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:53:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:53:39.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:53:39.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:53:39.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:53:39.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:53:39.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:53:39.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:53:39.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:53:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:53:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:53:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:53:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:53:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:53:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:53:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:53:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:53:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:53:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:53:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:53:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:53:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:53:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:54:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:54:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:54:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:54:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:54:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:54:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:54:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:54:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:54:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:54:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:54:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:54:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:54:39.665 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:54:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:54:39.674 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:54:39.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:54:39.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:54:39.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:54:39.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:54:39.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:54:39.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:54:39.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:54:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:54:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:54:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:54:49.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:54:49.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:54:49.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:54:49.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:54:49.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:54:49.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:54:49.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:54:49.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:54:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:54:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:54:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:55:09.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:55:09.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:55:09.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:55:09.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:55:09.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:55:09.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:55:09.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:55:09.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:55:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:55:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:55:09.786 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:55:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:55:39.668 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:55:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:55:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:55:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:55:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:55:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:55:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:55:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:55:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:55:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:55:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:55:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:55:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:55:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:55:49.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:55:49.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:55:49.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:55:49.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:55:49.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:55:49.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:55:49.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:55:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:55:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:55:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:56:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:56:09.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:56:09.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:56:09.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:56:09.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:56:09.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:56:09.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:56:09.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:56:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:56:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:56:09.788 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:56:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:56:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:56:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:56:39.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:56:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:56:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:56:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:56:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:56:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:56:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:56:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:56:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:56:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:56:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:56:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:56:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:56:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:56:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:56:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:56:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:56:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:56:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:56:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:56:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:56:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:57:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:57:09.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:57:09.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:57:09.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:57:09.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:57:09.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:57:09.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:57:09.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:57:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:57:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:57:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:57:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:57:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:57:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:57:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:57:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:57:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:57:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:57:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:57:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:57:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:57:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:57:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:57:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.053 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:57:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:57:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:57:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:57:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:57:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:57:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:57:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:57:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:57:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:57:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:57:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:57:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:58:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:58:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:58:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:58:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:58:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:58:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:58:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:58:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:58:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:58:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:58:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:58:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:58:39.671 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:58:39.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:58:39.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:58:39.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:58:39.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:58:39.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:58:39.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:58:39.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:58:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:58:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:58:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:58:49.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:58:49.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:58:49.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:58:49.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:58:49.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:58:49.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:58:49.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:58:49.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:58:49.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:58:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:58:49.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:59:09.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:59:09.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:59:09.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:59:09.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:59:09.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:59:09.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:59:09.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:59:09.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:59:09.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:59:09.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 19:59:09.778 139328 ERROR oslo_service.periodic_task 
>2018-06-24 19:59:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:59:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 19:59:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 19:59:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:59:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:59:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:59:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:59:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:59:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:59:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:59:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:59:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:59:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:59:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 19:59:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 19:59:49.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 19:59:49.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 19:59:49.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 19:59:49.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 19:59:49.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 19:59:49.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 19:59:49.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 19:59:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 19:59:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 19:59:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:00:09.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:00:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:00:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:00:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:00:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:00:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:00:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:00:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:00:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:00:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:00:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:00:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:00:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:00:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:00:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:00:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:00:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:00:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:00:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:00:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:00:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:00:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:00:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:00:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:00:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:00:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:00:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:00:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:00:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:00:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:00:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:00:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:00:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:00:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:00:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:00:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:01:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:01:09.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:01:09.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:01:09.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:01:09.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:01:09.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:01:09.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:01:09.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:01:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:01:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:01:09.796 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:01:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:01:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:01:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:01:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:01:39.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:01:39.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:01:39.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:01:39.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:01:39.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:01:39.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:01:39.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:01:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:01:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:01:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:01:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:01:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:01:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:01:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:01:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:01:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:01:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:01:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:01:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:01:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:01:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:02:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:02:09.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:02:09.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:02:09.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:02:09.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:02:09.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:02:09.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:02:09.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:02:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:02:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:02:09.791 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:02:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:02:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:02:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:02:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:02:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:02:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:02:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:02:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:02:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:02:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:02:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:02:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:02:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:02:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:02:49.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:02:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:02:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:02:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:02:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:02:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:02:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:02:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:02:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:02:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:02:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:03:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:03:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:03:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:03:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:03:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:03:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:03:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:03:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:03:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:03:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:03:09.795 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:03:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:03:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:03:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:03:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:03:39.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:03:39.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:03:39.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:03:39.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:03:39.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:03:39.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:03:39.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:03:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:03:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:03:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:03:49.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:03:49.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:03:49.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:03:49.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:03:49.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:03:49.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:03:49.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:03:49.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:03:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:03:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:03:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:04:09.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:04:09.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:04:09.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:04:09.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:04:09.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:04:09.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:04:09.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:04:09.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:04:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:04:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:04:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:04:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:04:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:04:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:04:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:04:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:04:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:04:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:04:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:04:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:04:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:04:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:04:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:04:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:04:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:04:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:04:49.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:04:49.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:04:49.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:04:49.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:04:49.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:04:49.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:04:49.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:04:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:04:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:04:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:05:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:05:09.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:05:09.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:05:09.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:05:09.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:05:09.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:05:09.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:05:09.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:05:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:05:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:05:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:05:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:05:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:05:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:05:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:05:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:05:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:05:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:05:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:05:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:05:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:05:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:05:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:05:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:05:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:05:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:05:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:05:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:05:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:05:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:05:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:05:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:05:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:05:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:05:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:05:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:06:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:06:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:06:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:06:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:06:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:06:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:06:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:06:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:06:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:06:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:06:09.795 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:06:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:06:39.668 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:06:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:06:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:06:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:06:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:06:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:06:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:06:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:06:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:06:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:06:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:06:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:06:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:06:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:06:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:06:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:06:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:06:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:06:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:06:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:06:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:06:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:06:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:06:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:07:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:07:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:07:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:07:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:07:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:07:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:07:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:07:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:07:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:07:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:07:09.789 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:07:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:07:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:07:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:07:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:07:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:07:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:07:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:07:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:07:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:07:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:07:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:07:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:07:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:07:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:07:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:07:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:07:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:07:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:07:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:07:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:07:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:07:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:07:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:07:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:07:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:08:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:08:09.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:08:09.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:08:09.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:08:09.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:08:09.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:08:09.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:08:09.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:08:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:08:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:08:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:08:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:08:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:08:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:08:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:08:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:08:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:08:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:08:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:08:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:08:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:08:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:08:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:08:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:08:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:08:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:08:49.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:08:49.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:08:49.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:08:49.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:08:49.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:08:49.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:08:49.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:08:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:08:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:08:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:09:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:09:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:09:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:09:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:09:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:09:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:09:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:09:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:09:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:09:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:09:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:09:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:09:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:09:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:09:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:09:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:09:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:09:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:09:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:09:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:09:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:09:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:09:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:09:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.054 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:09:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:09:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:09:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:09:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:09:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:09:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:09:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:09:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:09:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:09:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:09:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.089 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:09:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:10:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:10:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:10:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:10:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:10:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:10:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:10:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:10:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:10:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:10:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.127 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:10:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:10:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:10:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:10:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:10:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:10:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:10:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:10:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:10:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:10:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:10:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:10:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:10:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:10:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:10:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:10:49.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:10:49.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:10:49.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:10:49.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:10:49.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:10:49.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:10:49.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:10:49.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:10:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:10:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:10:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:11:09.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:11:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:11:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:11:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:11:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:11:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:11:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:11:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:11:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:11:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:11:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:11:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:11:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:11:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:11:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:11:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:11:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:11:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:11:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:11:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:11:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:11:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:11:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:11:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:11:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:11:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:11:49.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:11:49.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:11:49.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:11:49.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:11:49.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:11:49.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:11:49.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:11:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:11:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:11:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:12:09.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:12:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:12:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:12:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:12:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:12:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:12:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:12:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:12:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:12:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:12:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:12:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:12:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:12:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:12:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:12:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:12:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:12:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:12:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:12:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:12:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:12:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:12:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:12:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:12:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:12:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:12:49.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:12:49.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:12:49.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:12:49.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:12:49.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:12:49.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:12:49.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:12:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:12:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:12:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:13:09.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:13:09.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:13:09.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:13:09.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:13:09.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:13:09.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:13:09.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:13:09.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:13:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:13:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:13:09.788 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:13:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:13:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:13:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:13:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:13:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:13:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:13:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:13:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:13:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:13:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:13:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:13:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:13:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:13:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:13:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:13:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:13:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:13:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:13:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:13:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:13:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:13:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:13:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:13:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.09 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:13:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:14:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:14:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:14:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:14:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:14:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:14:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:14:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:14:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:14:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:14:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.127 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:14:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:14:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:14:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:14:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:14:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:14:39.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:14:39.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:14:39.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:14:39.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:14:39.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:14:39.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:14:39.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:14:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:14:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:14:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:14:49.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:14:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:14:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:14:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:14:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:14:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:14:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:14:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:14:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:14:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:14:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:15:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:15:09.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:15:09.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:15:09.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:15:09.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:15:09.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:15:09.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:15:09.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:15:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:15:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:15:09.786 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:15:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:15:39.668 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:15:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:15:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:15:39.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:15:39.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:15:39.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:15:39.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:15:39.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:15:39.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:15:39.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:15:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:15:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:15:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:15:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:15:49.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:15:49.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:15:49.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:15:49.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:15:49.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:15:49.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:15:49.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:15:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:15:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:15:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:16:09.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:16:09.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:16:09.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:16:09.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:16:09.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:16:09.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:16:09.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:16:09.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:16:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:16:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:16:09.786 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:16:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:16:39.668 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:16:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:16:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:16:39.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:16:39.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:16:39.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:16:39.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:16:39.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:16:39.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:16:39.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:16:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:16:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:16:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:16:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:16:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:16:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:16:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:16:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:16:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:16:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:16:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:16:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:16:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:16:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:17:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:17:09.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:17:09.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:17:09.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:17:09.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:17:09.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:17:09.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:17:09.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:17:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:17:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:17:09.785 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:17:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:17:39.668 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:17:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:17:39.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:17:39.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:17:39.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:17:39.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:17:39.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:17:39.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:17:39.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:17:39.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:17:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:17:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:17:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:17:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:17:49.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:17:49.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:17:49.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:17:49.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:17:49.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:17:49.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:17:49.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:17:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:17:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:17:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:18:09.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:18:09.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:18:09.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:18:09.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:18:09.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:18:09.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:18:09.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:18:09.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:18:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:18:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:18:09.787 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:18:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:18:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:18:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:18:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:18:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:18:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:18:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:18:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:18:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:18:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:18:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:18:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:18:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:18:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:18:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:18:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:18:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:18:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:18:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:18:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:18:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:18:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:18:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:18:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:18:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:19:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:19:09.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:19:09.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:19:09.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:19:09.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:19:09.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:19:09.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:19:09.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:19:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:19:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:19:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:19:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:19:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:19:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:19:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:19:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:19:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:19:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:19:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:19:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:19:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:19:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:19:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:19:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:19:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:19:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:19:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:19:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:19:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:19:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:19:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:19:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:19:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:19:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:19:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:19:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:20:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:20:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:20:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:20:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:20:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:20:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:20:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:20:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:20:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:20:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:20:09.795 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:20:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:20:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:20:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:20:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:20:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:20:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:20:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:20:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:20:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:20:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:20:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:20:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:20:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:20:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:20:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:20:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:20:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:20:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:20:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:20:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:20:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:20:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:20:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:20:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:20:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:21:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:21:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:21:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:21:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:21:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:21:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:21:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:21:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:21:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:21:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:21:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:21:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:21:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:21:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:21:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:21:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:21:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:21:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:21:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:21:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:21:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:21:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:21:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:21:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:21:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:21:49.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:21:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:21:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:21:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:21:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:21:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:21:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:21:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:21:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:21:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:21:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:22:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:22:09.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:22:09.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:22:09.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:22:09.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:22:09.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:22:09.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:22:09.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:22:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:22:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:22:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:22:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:22:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:22:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:22:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:22:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:22:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:22:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:22:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:22:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:22:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:22:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:22:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:22:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:22:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:22:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:22:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:22:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:22:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:22:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:22:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:22:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:22:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:22:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:22:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:22:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:23:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:23:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:23:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:23:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:23:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:23:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:23:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:23:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:23:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:23:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:23:09.794 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:23:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:23:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:23:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:23:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:23:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:23:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:23:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:23:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:23:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:23:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:23:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:23:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:23:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:23:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:23:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:23:49.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:23:49.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:23:49.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:23:49.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:23:49.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:23:49.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:23:49.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:23:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:23:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:23:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:24:09.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:24:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:24:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:24:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:24:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:24:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:24:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:24:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:24:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:24:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:24:09.796 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:24:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:24:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:24:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:24:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:24:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:24:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:24:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:24:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:24:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:24:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:24:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:24:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:24:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:24:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:24:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:24:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:24:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:24:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:24:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:24:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:24:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:24:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:24:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:24:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:24:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:25:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:25:09.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:25:09.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:25:09.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:25:09.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:25:09.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:25:09.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:25:09.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:25:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:25:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:25:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:25:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:25:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:25:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:25:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:25:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:25:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:25:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:25:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:25:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:25:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:25:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:25:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:25:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:25:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:25:49.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:25:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:25:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:25:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:25:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:25:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:25:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:25:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:25:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:25:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:25:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:26:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:26:09.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:26:09.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:26:09.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:26:09.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:26:09.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:26:09.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:26:09.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:26:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:26:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:26:09.796 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:26:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:26:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:26:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:26:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:26:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:26:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:26:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:26:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:26:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:26:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:26:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:26:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:26:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:26:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:26:49.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:26:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:26:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:26:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:26:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:26:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:26:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:26:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:26:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:26:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:26:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:27:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:27:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:27:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:27:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:27:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:27:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:27:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:27:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:27:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:27:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:27:09.799 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:27:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:27:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:27:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:27:39.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:27:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:27:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:27:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:27:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:27:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:27:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:27:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:27:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:27:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:27:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:27:49.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:27:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:27:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:27:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:27:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:27:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:27:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:27:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:27:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:27:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:27:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:28:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:28:09.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:28:09.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:28:09.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:28:09.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:28:09.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:28:09.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:28:09.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:28:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:28:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:28:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:28:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:28:39.686 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:28:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:28:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:28:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:28:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:28:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:28:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:28:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:28:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:28:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:28:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:28:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:28:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:28:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:28:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:28:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:28:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:28:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:28:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:28:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:28:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:28:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:28:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:28:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:29:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:29:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:29:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:29:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:29:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:29:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:29:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:29:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:29:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:29:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:29:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:29:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:29:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:29:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:29:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:29:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:29:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:29:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:29:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:29:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:29:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:29:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:29:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:29:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:29:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:29:49.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:29:49.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:29:49.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:29:49.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:29:49.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:29:49.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:29:49.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:29:49.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:29:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:29:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:29:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:30:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:30:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:30:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:30:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:30:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:30:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:30:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:30:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:30:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:30:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:30:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:30:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:30:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:30:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:30:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:30:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:30:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:30:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:30:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:30:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:30:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:30:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:30:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:30:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:30:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:30:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:30:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:30:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:30:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:30:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:30:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:30:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:30:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:30:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:30:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:30:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:31:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:31:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:31:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:31:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:31:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:31:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:31:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:31:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:31:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:31:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:31:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:31:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:31:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:32:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:32:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:32:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:32:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:32:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:32:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:32:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:32:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:32:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:32:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:32:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:32:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:32:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:32:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:32:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:32:49.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:32:49.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:32:49.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:32:49.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:32:49.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:32:49.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:32:49.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:32:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:32:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:32:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:33:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:33:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:33:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:33:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:33:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:33:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:33:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:33:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:33:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:33:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:33:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:33:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:33:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:33:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:33:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:33:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:33:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:33:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:33:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:33:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:33:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:33:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:33:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:33:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:33:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:33:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:33:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:33:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:33:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:33:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:33:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:33:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:33:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:33:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:33:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:33:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:34:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:34:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:34:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:34:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:34:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:34:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:34:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:34:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:34:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:34:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:34:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:34:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:34:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:34:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:34:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:34:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:34:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:34:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:34:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:34:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:34:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:34:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:34:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:34:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:34:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:34:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:34:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:34:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:34:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:34:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:34:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:34:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:34:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:34:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:34:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:34:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:35:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:35:09.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:35:09.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:35:09.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:35:09.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:35:09.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:35:09.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:35:09.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:35:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:35:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:35:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:35:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:35:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:35:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:35:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:35:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:35:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:35:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:35:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:35:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:35:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:35:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:35:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:35:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:35:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:35:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:35:49.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:35:49.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:35:49.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:35:49.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:35:49.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:35:49.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:35:49.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:35:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:35:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:35:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:36:09.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:36:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:36:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:36:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:36:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:36:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:36:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:36:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:36:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:36:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:36:09.808 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:36:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:36:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:36:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:36:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:36:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:36:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:36:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:36:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:36:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:36:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:36:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:36:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:36:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:36:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:36:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:36:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:36:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:36:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:36:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:36:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:36:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:36:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:36:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:36:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:36:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:37:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:37:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:37:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:37:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:37:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:37:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:37:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:37:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:37:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:37:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:37:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:37:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:37:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:37:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:37:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:37:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:37:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:37:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:37:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:37:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:37:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:37:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:37:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:37:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:37:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:37:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:37:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:37:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:37:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:37:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:37:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:37:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:37:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:37:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:37:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:37:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:38:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:38:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:38:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:38:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:38:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:38:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:38:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:38:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:38:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:38:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:38:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:38:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:38:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:38:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:38:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:38:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:38:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:38:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:38:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:38:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:38:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:38:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:38:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:38:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:38:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:38:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:38:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:38:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:38:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:38:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:38:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:38:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:38:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:38:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:38:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:38:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:39:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:39:09.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:39:09.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:39:09.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:39:09.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:39:09.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:39:09.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:39:09.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:39:09.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:39:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:39:09.819 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:39:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:39:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:39:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:39:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:39:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:39:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:39:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:39:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:39:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:39:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:39:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:39:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:39:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:39:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:39:49.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:39:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:39:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:39:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:39:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:39:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:39:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:39:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:39:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:39:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:39:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:40:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:40:09.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:40:09.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:40:09.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:40:09.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:40:09.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:40:09.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:40:09.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:40:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:40:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:40:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:40:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:40:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:40:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:40:39.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:40:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:40:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:40:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:40:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:40:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:40:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:40:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:40:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:40:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:40:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:40:49.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:40:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:40:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:40:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:40:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:40:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:40:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:40:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:40:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:40:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:40:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:41:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:41:09.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:41:09.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:41:09.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:41:09.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:41:09.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:41:09.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:41:09.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:41:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:41:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:41:09.819 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:41:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:41:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:41:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:41:39.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:41:39.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:41:39.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:41:39.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:41:39.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:41:39.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:41:39.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:41:39.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:41:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:41:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:41:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:41:49.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:41:49.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:41:49.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:41:49.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:41:49.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:41:49.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:41:49.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:41:49.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:41:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:41:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:41:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:42:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:42:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:42:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:42:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:42:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:42:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:42:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:42:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:42:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:42:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:42:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:42:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:42:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:42:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:42:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:42:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:42:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:42:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:42:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:42:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:42:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:42:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:42:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:42:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:42:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:42:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:42:49.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:42:49.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:42:49.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:42:49.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:42:49.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:42:49.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:42:49.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:42:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:42:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.091 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:42:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:43:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:43:09.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:43:09.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:43:09.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:43:09.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:43:09.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:43:09.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:43:09.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:43:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:43:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:43:09.840 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:43:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:43:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:43:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:43:39.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:43:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:43:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:43:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:43:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:43:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:43:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:43:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:43:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:43:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:43:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:43:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:43:49.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:43:49.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:43:49.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:43:49.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:43:49.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:43:49.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:43:49.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:43:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:43:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:43:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:44:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:44:09.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:44:09.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:44:09.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:44:09.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:44:09.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:44:09.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:44:09.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:44:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:44:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:44:09.827 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:44:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:44:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:44:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:44:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:44:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:44:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:44:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:44:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:44:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:44:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:44:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:44:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:44:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:44:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:44:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:44:49.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:44:49.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:44:49.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:44:49.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:44:49.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:44:49.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:44:49.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:44:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:44:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:44:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:45:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:45:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:45:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:45:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:45:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:45:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:45:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:45:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:45:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:45:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:45:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:45:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:45:39.697 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:45:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:45:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:45:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:45:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:45:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:45:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:45:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:45:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:45:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:45:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:45:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:45:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:45:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:45:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:45:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:45:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:45:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:45:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:45:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:45:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:45:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:45:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:45:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:46:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:46:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:46:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:46:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:46:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:46:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:46:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:46:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:46:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:46:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:46:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:46:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:46:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:46:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:46:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:46:39.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:46:39.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:46:39.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:46:39.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:46:39.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:46:39.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:46:39.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:46:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:46:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:46:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:46:49.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:46:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:46:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:46:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:46:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:46:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:46:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:46:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:46:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:46:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:46:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:47:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:47:09.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:47:09.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:47:09.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:47:09.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:47:09.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:47:09.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:47:09.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:47:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:47:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:47:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:47:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:47:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:47:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:47:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:47:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:47:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:47:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:47:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:47:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:47:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:47:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:47:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:47:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:47:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:47:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:47:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:47:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:47:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:47:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:47:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:47:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:47:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:47:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:47:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:47:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:48:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:48:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:48:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:48:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:48:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:48:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:48:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:48:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:48:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:48:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:48:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:48:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:48:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:48:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:48:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:48:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:48:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:48:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:48:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:48:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:48:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:48:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:48:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:48:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:48:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:48:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:48:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:48:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:48:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:48:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:48:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:48:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:48:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:48:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:48:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:48:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:49:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:49:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:49:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:49:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:49:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:49:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:49:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:49:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:49:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:49:09.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:49:09.818 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:49:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:49:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:49:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:49:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:49:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:49:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:49:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:49:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:49:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:49:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:49:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:49:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:49:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:49:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:49:49.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:49:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:49:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:49:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:49:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:49:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:49:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:49:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:49:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:49:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:49:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:50:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:50:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:50:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:50:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:50:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:50:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:50:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:50:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:50:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:50:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:50:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:50:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:50:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:50:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:50:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:50:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:50:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:50:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:50:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:50:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:50:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:50:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:50:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:50:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:50:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:50:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:50:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:50:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:50:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:50:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:50:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:50:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:50:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:50:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:50:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:50:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:51:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:51:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:51:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:51:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:51:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:51:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:51:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:51:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:51:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:51:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:51:09.808 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:51:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:51:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:51:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:51:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:51:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:51:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:51:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:51:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:51:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:51:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:51:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:51:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:51:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:51:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:51:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:51:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:51:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:51:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:51:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:51:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:51:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:51:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:51:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:51:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:51:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:52:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:52:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:52:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:52:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:52:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:52:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:52:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:52:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:52:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:52:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:52:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:52:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:52:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:52:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:52:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:52:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:52:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:52:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:52:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:52:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:52:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:52:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:52:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:52:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:52:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:52:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:52:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:52:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:52:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:52:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:52:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:52:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:52:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:52:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:52:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:52:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:53:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:53:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:53:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:53:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:53:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:53:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:53:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:53:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:53:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:53:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:53:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:53:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:53:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:53:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:53:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:53:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:53:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:53:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:53:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:53:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:53:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:53:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:53:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:53:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:53:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:53:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:53:49.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:53:49.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:53:49.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:53:49.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:53:49.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:53:49.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:53:49.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:53:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:53:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:53:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:54:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:54:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:54:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:54:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:54:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:54:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:54:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:54:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:54:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:54:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:54:09.808 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:54:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:54:39.691 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:54:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:54:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:54:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:54:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:54:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:54:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:54:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:54:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:54:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:54:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:54:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:54:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:54:49.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:54:49.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:54:49.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:54:49.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:54:49.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:54:49.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:54:49.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:54:49.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:54:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:54:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:54:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:55:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:55:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:55:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:55:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:55:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:55:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:55:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:55:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:55:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:55:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:55:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:55:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:55:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:55:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:55:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:55:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:55:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:55:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:55:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:55:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:55:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:55:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:55:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:55:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:55:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:55:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:55:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:55:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:55:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:55:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:55:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:55:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:55:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:55:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:55:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:55:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:56:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:56:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:56:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:56:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:56:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:56:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:56:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:56:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:56:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:56:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:56:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:56:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:56:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:56:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:56:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:56:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:56:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:56:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:56:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:56:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:56:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:56:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:56:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:56:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:56:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:56:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:56:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:56:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:56:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:56:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:56:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:56:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:56:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:56:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:56:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:56:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:57:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:57:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:57:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:57:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:57:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:57:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:57:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:57:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:57:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:57:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:57:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:57:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:57:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:57:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:57:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:57:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:57:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:57:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:57:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:57:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:57:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:57:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:57:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:57:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:57:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:57:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:57:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:57:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:57:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:57:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:57:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:57:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:57:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:57:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:57:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:57:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:58:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:58:09.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:58:09.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:58:09.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:58:09.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:58:09.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:58:09.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:58:09.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:58:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:58:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:58:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:58:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:58:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:58:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:58:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:58:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:58:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:58:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:58:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:58:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:58:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:58:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:58:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:58:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:58:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:58:49.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:58:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:58:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:58:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:58:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:58:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:58:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:58:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:58:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:58:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:58:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:59:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:59:09.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:59:09.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:59:09.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:59:09.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:59:09.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:59:09.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:59:09.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:59:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:59:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 20:59:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-24 20:59:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:59:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 20:59:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 20:59:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:59:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:59:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:59:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:59:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:59:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:59:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:59:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:59:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:59:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:59:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 20:59:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 20:59:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 20:59:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 20:59:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 20:59:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 20:59:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 20:59:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 20:59:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 20:59:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 20:59:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 20:59:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:00:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:00:09.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:00:09.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:00:09.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:00:09.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:00:09.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:00:09.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:00:09.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:00:09.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:00:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:00:09.826 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:00:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:00:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:00:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:00:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:00:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:00:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:00:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:00:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:00:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:00:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:00:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:00:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:00:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:00:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:00:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:00:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:00:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:00:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:00:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:00:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:00:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:00:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:00:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:00:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:00:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:01:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:01:09.818 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:01:09.818 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:01:09.818 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:01:09.818 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:01:09.818 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:01:09.818 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:01:09.818 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:01:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:01:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:01:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:01:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:01:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:01:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:01:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:01:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:01:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:01:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:01:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:01:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:01:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:01:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:01:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:01:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:01:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:01:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:01:49.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:01:49.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:01:49.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:01:49.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:01:49.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:01:49.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:01:49.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:01:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:01:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:01:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:02:09.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:02:09.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:02:09.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:02:09.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:02:09.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:02:09.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:02:09.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:02:09.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:02:09.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:02:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:02:09.819 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:02:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:02:39.691 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:02:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:02:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:02:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:02:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:02:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:02:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:02:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:02:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:02:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:02:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:02:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:02:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:02:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:02:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:02:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:02:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:02:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:02:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:02:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:02:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:02:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:02:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:02:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:03:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:03:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:03:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:03:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:03:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:03:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:03:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:03:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:03:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:03:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:03:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:03:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:03:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:03:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:03:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:03:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:03:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:03:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:03:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:03:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:03:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:03:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:03:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:03:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:03:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:03:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:03:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:03:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:03:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:03:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:03:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:03:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:03:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:03:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:03:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:03:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:04:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:04:09.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:04:09.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:04:09.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:04:09.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:04:09.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:04:09.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:04:09.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:04:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:04:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:04:09.818 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:04:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:04:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:04:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:04:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:04:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:04:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:04:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:04:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:04:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:04:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:04:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:04:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:04:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:04:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:04:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:04:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:04:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:04:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:04:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:04:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:04:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:04:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:04:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:04:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:04:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:05:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:05:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:05:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:05:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:05:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:05:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:05:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:05:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:05:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:05:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:05:09.805 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:05:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:05:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:05:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:05:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:05:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:05:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:05:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:05:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:05:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:05:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:05:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:05:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:05:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:05:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:05:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:05:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:05:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:05:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:05:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:05:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:05:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:05:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:05:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:05:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:05:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:06:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:06:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:06:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:06:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:06:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:06:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:06:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:06:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:06:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:06:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:06:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:06:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:06:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:06:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:06:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:06:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:06:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:06:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:06:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:06:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:06:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:06:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:06:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:06:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:06:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:06:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:06:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:06:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:06:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:06:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:06:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:06:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:06:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:06:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:07:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:07:09.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:07:09.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:07:09.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:07:09.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:07:09.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:07:09.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:07:09.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:07:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:07:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:07:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:07:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:07:39.692 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:07:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:07:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:07:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:07:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:07:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:07:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:07:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:07:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:07:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:07:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:07:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:07:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:07:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:07:49.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:07:49.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:07:49.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:07:49.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:07:49.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:07:49.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:07:49.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:07:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:07:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:07:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:08:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:08:09.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:08:09.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:08:09.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:08:09.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:08:09.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:08:09.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:08:09.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:08:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:08:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.127 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:08:09.827 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:08:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:08:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:08:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:08:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:08:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:08:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:08:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:08:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:08:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:08:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:08:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:08:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:08:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:08:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:08:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:08:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:08:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:08:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:08:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:08:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:08:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:08:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:08:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:08:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:08:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:09:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:09:09.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:09:09.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:09:09.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:09:09.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:09:09.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:09:09.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:09:09.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:09:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:09:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:09:09.814 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:09:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:09:39.686 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:09:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:09:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:09:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:09:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:09:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:09:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:09:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:09:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:09:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:09:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:09:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:09:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:09:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:09:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:09:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:09:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:09:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:09:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:09:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:09:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:09:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:09:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:09:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:10:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:10:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:10:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:10:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:10:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:10:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:10:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:10:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:10:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:10:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:10:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:10:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:10:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:10:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:10:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:10:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:10:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:10:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:10:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:10:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:10:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:10:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:10:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:10:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:10:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:10:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:10:49.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:10:49.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:10:49.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:10:49.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:10:49.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:10:49.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:10:49.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:10:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:10:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:10:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:11:09.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:11:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:11:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:11:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:11:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:11:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:11:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:11:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:11:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:11:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:11:09.802 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:11:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:11:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:11:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:11:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:11:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:11:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:11:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:11:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:11:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:11:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:11:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:11:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:11:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:11:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:11:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:11:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:11:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:11:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:11:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:11:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:11:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:11:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:11:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:11:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:11:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:12:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:12:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:12:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:12:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:12:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:12:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:12:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:12:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:12:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:12:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:12:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:12:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:12:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:12:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:12:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:12:39.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:12:39.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:12:39.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:12:39.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:12:39.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:12:39.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:12:39.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:12:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:12:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:12:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:12:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:12:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:12:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:12:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:12:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:12:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:12:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:12:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:12:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:12:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:12:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:13:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:13:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:13:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:13:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:13:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:13:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:13:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:13:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:13:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:13:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:13:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:13:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:13:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:13:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:13:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:13:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:13:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:13:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:13:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:13:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:13:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:13:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:13:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:13:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:13:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:13:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:13:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:13:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:13:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:13:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:13:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:13:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:13:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:13:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:13:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:13:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:14:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:14:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:14:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:14:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:14:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:14:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:14:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:14:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:14:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:14:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:14:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:14:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:14:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:14:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:14:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:14:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:14:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:14:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:14:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:14:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:14:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:14:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:14:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:14:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:14:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:14:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:14:49.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:14:49.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:14:49.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:14:49.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:14:49.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:14:49.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:14:49.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:14:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:14:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:14:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:15:09.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:15:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:15:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:15:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:15:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:15:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:15:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:15:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:15:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:15:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:15:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:15:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:15:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:15:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:15:39.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:15:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:15:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:15:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:15:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:15:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:15:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:15:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:15:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:15:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:15:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:15:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:15:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:15:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:15:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:15:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:15:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:15:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:15:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:15:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:15:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:15:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:16:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:16:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:16:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:16:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:16:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:16:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:16:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:16:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:16:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:16:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:16:09.805 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:16:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:16:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:16:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:16:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:16:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:16:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:16:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:16:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:16:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:16:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:16:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:16:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:16:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:16:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:16:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:16:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:16:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:16:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:16:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:16:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:16:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:16:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:16:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:16:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:16:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:17:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:17:09.801 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:17:09.801 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:17:09.801 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:17:09.801 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:17:09.801 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:17:09.801 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:17:09.801 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:17:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:17:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:17:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:17:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:17:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:17:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:17:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:17:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:17:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:17:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:17:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:17:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:17:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:17:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:17:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:17:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:17:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:17:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:17:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:17:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:17:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:17:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:17:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:17:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:17:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:17:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:17:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:17:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:18:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:18:09.801 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:18:09.801 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:18:09.801 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:18:09.801 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:18:09.801 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:18:09.801 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:18:09.801 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:18:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:18:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:18:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:18:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:18:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:18:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:18:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:18:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:18:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:18:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:18:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:18:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:18:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:18:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:18:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:18:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:18:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:18:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:18:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:18:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:18:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:18:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:18:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:18:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:18:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:18:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:18:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:18:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:19:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:19:09.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:19:09.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:19:09.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:19:09.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:19:09.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:19:09.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:19:09.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:19:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:19:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:19:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:19:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:19:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:19:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:19:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:19:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:19:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:19:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:19:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:19:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:19:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:19:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:19:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:19:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:19:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:19:49.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:19:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:19:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:19:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:19:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:19:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:19:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:19:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:19:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:19:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:19:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:20:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:20:09.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:20:09.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:20:09.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:20:09.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:20:09.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:20:09.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:20:09.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:20:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:20:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:20:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:20:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:20:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:20:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:20:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:20:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:20:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:20:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:20:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:20:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:20:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:20:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:20:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:20:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:20:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:20:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:20:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:20:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:20:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:20:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:20:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:20:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:20:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:20:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:20:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:20:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:21:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:21:09.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:21:09.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:21:09.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:21:09.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:21:09.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:21:09.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:21:09.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:21:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:21:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:21:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:21:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:21:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:21:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:21:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:21:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:21:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:21:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:21:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:21:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:21:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:21:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:21:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:21:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:21:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:21:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:21:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:21:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:21:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:21:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:21:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:21:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:21:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:21:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:21:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:21:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:22:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:22:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:22:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:22:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:22:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:22:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:22:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:22:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:22:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:22:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:22:09.805 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:22:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:22:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:22:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:22:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:22:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:22:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:22:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:22:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:22:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:22:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:22:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:22:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:22:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.053 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:22:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:22:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:22:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:22:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:22:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:22:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:22:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:22:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:22:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:22:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:22:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.089 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:22:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:23:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:23:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:23:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:23:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:23:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:23:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:23:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:23:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:23:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:23:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:23:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:23:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:23:39.686 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:23:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:23:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:23:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:23:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:23:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:23:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:23:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:23:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:23:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:23:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:23:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:23:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:23:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:23:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:23:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:23:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:23:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:23:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:23:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:23:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:23:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:23:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:23:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:24:09.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:24:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:24:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:24:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:24:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:24:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:24:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:24:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:24:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:24:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:24:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:24:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:24:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:24:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:24:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:24:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:24:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:24:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:24:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:24:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:24:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:24:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:24:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:24:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:24:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:24:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:24:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:24:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:24:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:24:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:24:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:24:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:24:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:24:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:24:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:24:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:25:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:25:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:25:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:25:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:25:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:25:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:25:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:25:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:25:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:25:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:25:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:25:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:25:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:25:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:25:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:25:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:25:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:25:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:25:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:25:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:25:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:25:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:25:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:25:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:25:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:25:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:25:49.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:25:49.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:25:49.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:25:49.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:25:49.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:25:49.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:25:49.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:25:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:25:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:25:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:26:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:26:09.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:26:09.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:26:09.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:26:09.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:26:09.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:26:09.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:26:09.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:26:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:26:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:26:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:26:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:26:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:26:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:26:39.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:26:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:26:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:26:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:26:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:26:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:26:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:26:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:26:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:26:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:26:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:26:49.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:26:49.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:26:49.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:26:49.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:26:49.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:26:49.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:26:49.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:26:49.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:26:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:26:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:26:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:27:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:27:09.801 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:27:09.801 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:27:09.801 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:27:09.801 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:27:09.801 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:27:09.801 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:27:09.801 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:27:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:27:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:27:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:27:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:27:39.686 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:27:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:27:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:27:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:27:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:27:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:27:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:27:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:27:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:27:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:27:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:27:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:27:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:27:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:27:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:27:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:27:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:27:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:27:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:27:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:27:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:27:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:27:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:27:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:28:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:28:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:28:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:28:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:28:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:28:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:28:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:28:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:28:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:28:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:28:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:28:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:28:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:28:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:28:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:28:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:28:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:28:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:28:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:28:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:28:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:28:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:28:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:28:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:28:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:28:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:28:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:28:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:28:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:28:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:28:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:28:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:28:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:28:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:28:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:28:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:29:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:29:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:29:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:29:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:29:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:29:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:29:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:29:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:29:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:29:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:29:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:29:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:29:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:29:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:29:39.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:29:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:29:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:29:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:29:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:29:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:29:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:29:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:29:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:29:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:29:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:29:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:29:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:29:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:29:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:29:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:29:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:29:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:29:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:29:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:29:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:29:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:30:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:30:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:30:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:30:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:30:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:30:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:30:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:30:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:30:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:30:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:30:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:30:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:30:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:30:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:30:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:30:39.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:30:39.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:30:39.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:30:39.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:30:39.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:30:39.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:30:39.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:30:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:30:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:30:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:30:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:30:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:30:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:30:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:30:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:30:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:30:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:30:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:30:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:30:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:30:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:31:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:31:09.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:31:09.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:31:09.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:31:09.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:31:09.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:31:09.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:31:09.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:31:09.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:31:09.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:31:09.818 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:31:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:31:39.691 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:31:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:31:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:31:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:31:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:31:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:31:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:31:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:31:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:31:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:31:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:31:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:31:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:31:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:31:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:31:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:31:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:31:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:31:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:31:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:31:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:31:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:31:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:31:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:32:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:32:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:32:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:32:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:32:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:32:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:32:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:32:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:32:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:32:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:32:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:32:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:32:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:32:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:32:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:32:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:32:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:32:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:32:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:32:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:32:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:32:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:32:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:32:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:32:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:32:49.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:32:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:32:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:32:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:32:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:32:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:32:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:32:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:32:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:32:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:32:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:33:09.781 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:33:09.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:33:09.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:33:09.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:33:09.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:33:09.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:33:09.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:33:09.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:33:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:33:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:33:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:33:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:33:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:33:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:33:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:33:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:33:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:33:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:33:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:33:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:33:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:33:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:33:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:33:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:33:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:33:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:33:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:33:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:33:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:33:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:33:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:33:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:33:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:33:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:33:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:33:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:34:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:34:09.834 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:34:09.834 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:34:09.834 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:34:09.834 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:34:09.834 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:34:09.834 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:34:09.834 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:34:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:34:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.14 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:34:09.837 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:34:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:34:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:34:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:34:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:34:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:34:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:34:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:34:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:34:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:34:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:34:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:34:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:34:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:34:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:34:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:34:49.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:34:49.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:34:49.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:34:49.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:34:49.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:34:49.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:34:49.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:34:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:34:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:34:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:35:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:35:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:35:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:35:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:35:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:35:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:35:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:35:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:35:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:35:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:35:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:35:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:35:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:36:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:36:39.692 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:36:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:36:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:36:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:36:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:36:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:36:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:36:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:36:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:36:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:36:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:36:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:36:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:36:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:36:49.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:36:49.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:36:49.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:36:49.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:36:49.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:36:49.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:36:49.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:36:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:36:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:36:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:37:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:37:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:37:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:37:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:37:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:37:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:37:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:37:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:37:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:37:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:37:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:37:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:37:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:37:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:37:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:37:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:37:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:37:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:37:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:37:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:37:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:37:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:37:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:37:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:37:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:37:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:37:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:37:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:37:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:37:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:37:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:37:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:37:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:37:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:37:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:37:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:38:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:38:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:38:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:38:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:38:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:38:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:38:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:38:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:38:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:38:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:38:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:38:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:38:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:38:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:38:39.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:38:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:38:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:38:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:38:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:38:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:38:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:38:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:38:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:38:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:38:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:38:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:38:49.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:38:49.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:38:49.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:38:49.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:38:49.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:38:49.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:38:49.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:38:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:38:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:38:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:39:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:39:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:39:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:39:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:39:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:39:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:39:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:39:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:39:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:39:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:39:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:39:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:39:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:39:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:39:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:39:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:39:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:39:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:39:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:39:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:39:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:39:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:39:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:39:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:39:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:39:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:39:49.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:39:49.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:39:49.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:39:49.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:39:49.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:39:49.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:39:49.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:39:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:39:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:39:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:40:09.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:40:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:40:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:40:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:40:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:40:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:40:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:40:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:40:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:40:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:40:09.808 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:40:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:40:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:40:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:40:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:40:39.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:40:39.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:40:39.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:40:39.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:40:39.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:40:39.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:40:39.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:40:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:40:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:40:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:40:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:40:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:40:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:40:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:40:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:40:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:40:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:40:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:40:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:40:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:40:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:41:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:41:09.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:41:09.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:41:09.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:41:09.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:41:09.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:41:09.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:41:09.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:41:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:41:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:41:09.801 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:41:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:41:39.684 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:41:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:41:39.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:41:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:41:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:41:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:41:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:41:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:41:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:41:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:41:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:41:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:41:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:41:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:41:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:41:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:41:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:41:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:41:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:41:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:41:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:41:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:41:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:41:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:42:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:42:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:42:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:42:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:42:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:42:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:42:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:42:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:42:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:42:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:42:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:42:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:42:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:42:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:42:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:42:39.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:42:39.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:42:39.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:42:39.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:42:39.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:42:39.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:42:39.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:42:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:42:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:42:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:42:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:42:49.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:42:49.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:42:49.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:42:49.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:42:49.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:42:49.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:42:49.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:42:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:42:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:42:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:43:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:43:09.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:43:09.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:43:09.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:43:09.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:43:09.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:43:09.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:43:09.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:43:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:43:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:43:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:43:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:43:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:43:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:43:39.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:43:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:43:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:43:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:43:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:43:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:43:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:43:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:43:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:43:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:43:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:43:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:43:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:43:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:43:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:43:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:43:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:43:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:43:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:43:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:43:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:43:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:44:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:44:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:44:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:44:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:44:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:44:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:44:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:44:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:44:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:44:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:44:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:44:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:44:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:44:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:44:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:44:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:44:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:44:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:44:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:44:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:44:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:44:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:44:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:44:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:44:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:44:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:44:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:44:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:44:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:44:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:44:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:44:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:44:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:44:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:44:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:44:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:45:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:45:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:45:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:45:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:45:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:45:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:45:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:45:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:45:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:45:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:45:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:45:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:45:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:45:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:45:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:45:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:45:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:45:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:45:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:45:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:45:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:45:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:45:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:45:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.052 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:45:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:45:49.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:45:49.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:45:49.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:45:49.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:45:49.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:45:49.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:45:49.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:45:49.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:45:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:45:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.092 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:45:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:46:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:46:09.845 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:46:09.845 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:46:09.845 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:46:09.845 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:46:09.845 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:46:09.845 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:46:09.845 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:46:09.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:46:09.848 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.15 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:46:09.848 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:46:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:46:39.691 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:46:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:46:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:46:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:46:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:46:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:46:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:46:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:46:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:46:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:46:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:46:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:46:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:46:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:46:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:46:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:46:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:46:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:46:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:46:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:46:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:46:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:46:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:46:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:47:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:47:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:47:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:47:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:47:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:47:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:47:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:47:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:47:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:47:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:47:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:47:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:47:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:47:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:47:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:47:39.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:47:39.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:47:39.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:47:39.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:47:39.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:47:39.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:47:39.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:47:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:47:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.054 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:47:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:47:49.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:47:49.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:47:49.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:47:49.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:47:49.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:47:49.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:47:49.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:47:49.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:47:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:47:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.091 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:47:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:48:09.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:48:09.832 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:48:09.832 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:48:09.832 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:48:09.832 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:48:09.832 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:48:09.832 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:48:09.832 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:48:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:48:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.128 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:48:09.835 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:48:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:48:39.697 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:48:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:48:39.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:48:39.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:48:39.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:48:39.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:48:39.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:48:39.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:48:39.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:48:39.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:48:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:48:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:48:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:48:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:48:49.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:48:49.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:48:49.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:48:49.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:48:49.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:48:49.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:48:49.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:48:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:48:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:48:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:49:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:49:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:49:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:49:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:49:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:49:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:49:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:49:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:49:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:49:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:49:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:49:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:49:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:49:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:49:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:49:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:49:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:49:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:49:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:49:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:49:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:49:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:49:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:49:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:49:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:49:49.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:49:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:49:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:49:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:49:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:49:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:49:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:49:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:49:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:49:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:49:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:50:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:50:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:50:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:50:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:50:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:50:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:50:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:50:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:50:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:50:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:50:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:50:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:50:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:50:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:50:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:50:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:50:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:50:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:50:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:50:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:50:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:50:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:50:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:50:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:50:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:50:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:50:49.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:50:49.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:50:49.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:50:49.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:50:49.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:50:49.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:50:49.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:50:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:50:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.092 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:50:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:51:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:51:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:51:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:51:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:51:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:51:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:51:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:51:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:51:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:51:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:51:09.839 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:51:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:51:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:51:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:51:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:51:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:51:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:51:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:51:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:51:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:51:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:51:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:51:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:51:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:51:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:51:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:51:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:51:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:51:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:51:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:51:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:51:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:51:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:51:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:51:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:51:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:52:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:52:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:52:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:52:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:52:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:52:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:52:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:52:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:52:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:52:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:52:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:52:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:52:39.704 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:52:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:52:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:52:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:52:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:52:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:52:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:52:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:52:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:52:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:52:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:52:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:52:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:52:49.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:52:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:52:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:52:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:52:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:52:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:52:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:52:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:52:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:52:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:52:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:53:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:53:09.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:53:09.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:53:09.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:53:09.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:53:09.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:53:09.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:53:09.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:53:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:53:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:53:09.830 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:53:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:53:39.703 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:53:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:53:39.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:53:39.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:53:39.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:53:39.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:53:39.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:53:39.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:53:39.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:53:39.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:53:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:53:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:53:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:53:49.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:53:49.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:53:49.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:53:49.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:53:49.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:53:49.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:53:49.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:53:49.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:53:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:53:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:53:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:54:09.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:54:09.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:54:09.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:54:09.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:54:09.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:54:09.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:54:09.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:54:09.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:54:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:54:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:54:09.827 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:54:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:54:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:54:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:54:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:54:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:54:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:54:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:54:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:54:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:54:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:54:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:54:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:54:49.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:54:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:54:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:54:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:54:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:54:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:54:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:54:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:54:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:54:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:54:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:55:09.781 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:55:09.833 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:55:09.833 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:55:09.833 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:55:09.833 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:55:09.833 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:55:09.833 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:55:09.833 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:55:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:55:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.135 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:55:09.836 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:55:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:55:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:55:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:55:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:55:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:55:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:55:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:55:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:55:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:55:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:55:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:55:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:55:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:55:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:55:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:55:49.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:55:49.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:55:49.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:55:49.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:55:49.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:55:49.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:55:49.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:55:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:55:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:55:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:56:09.784 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:56:09.818 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:56:09.818 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:56:09.818 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:56:09.818 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:56:09.818 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:56:09.818 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:56:09.818 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:56:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:56:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:56:09.821 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:56:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:56:39.703 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:56:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:56:39.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:56:39.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:56:39.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:56:39.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:56:39.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:56:39.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:56:39.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:56:39.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:56:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:56:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:56:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:56:49.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:56:49.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:56:49.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:56:49.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:56:49.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:56:49.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:56:49.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:56:49.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:56:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:56:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:56:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:57:09.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:57:09.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:57:09.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:57:09.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:57:09.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:57:09.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:57:09.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:57:09.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:57:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:57:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:57:09.840 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:57:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:57:39.703 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:57:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:57:39.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:57:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:57:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:57:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:57:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:57:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:57:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:57:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:57:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:57:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:57:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:57:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:57:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:57:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:57:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:57:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:57:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:57:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:57:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:57:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:57:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:57:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:58:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:58:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:58:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:58:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:58:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:58:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:58:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:58:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:58:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:58:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:58:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:58:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:58:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:58:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:58:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:58:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:58:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:58:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:58:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:58:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:58:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:58:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:58:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:58:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:58:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:58:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:58:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:58:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:58:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:58:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:58:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:58:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:58:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:58:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:58:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:58:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:59:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:59:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:59:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:59:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:59:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:59:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:59:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:59:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:59:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:59:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 21:59:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-24 21:59:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:59:39.705 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 21:59:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 21:59:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:59:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:59:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:59:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:59:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:59:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:59:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:59:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:59:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:59:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:59:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 21:59:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 21:59:49.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 21:59:49.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 21:59:49.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 21:59:49.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 21:59:49.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 21:59:49.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 21:59:49.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 21:59:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 21:59:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 21:59:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:00:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:00:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:00:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:00:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:00:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:00:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:00:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:00:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:00:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:00:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:00:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:00:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:00:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:00:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:00:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:00:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:00:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:00:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:00:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:00:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:00:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:00:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:00:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:00:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:00:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:00:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:00:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:00:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:00:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:00:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:00:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:00:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:00:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:00:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:00:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:00:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:01:09.784 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:01:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:01:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:01:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:01:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:01:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:01:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:01:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:01:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:01:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:01:09.821 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:01:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:01:39.704 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:01:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:01:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:01:39.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:01:39.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:01:39.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:01:39.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:01:39.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:01:39.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:01:39.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:01:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:01:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:01:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:01:49.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:01:49.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:01:49.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:01:49.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:01:49.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:01:49.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:01:49.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:01:49.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:01:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:01:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:01:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:02:09.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:02:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:02:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:02:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:02:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:02:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:02:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:02:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:02:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:02:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:02:09.827 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:02:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:02:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:02:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:02:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:02:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:02:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:02:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:02:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:02:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:02:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:02:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:02:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:02:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:02:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:02:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:02:49.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:02:49.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:02:49.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:02:49.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:02:49.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:02:49.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:02:49.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:02:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:02:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.092 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:02:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:03:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:03:09.835 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:03:09.835 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:03:09.835 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:03:09.835 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:03:09.835 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:03:09.835 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:03:09.835 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:03:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:03:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:03:09.838 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:03:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:03:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:03:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:03:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:03:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:03:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:03:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:03:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:03:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:03:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:03:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:03:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:03:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:03:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:03:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:03:49.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:03:49.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:03:49.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:03:49.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:03:49.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:03:49.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:03:49.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:03:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:03:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:03:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:04:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:04:09.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:04:09.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:04:09.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:04:09.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:04:09.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:04:09.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:04:09.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:04:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:04:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:04:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:04:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:04:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:04:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:04:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:04:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:04:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:04:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:04:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:04:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:04:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:04:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:04:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:04:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:04:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:04:49.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:04:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:04:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:04:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:04:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:04:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:04:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:04:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:04:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:04:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:04:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:05:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:05:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:05:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:05:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:05:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:05:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:05:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:05:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:05:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:05:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:05:09.821 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:05:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:05:39.704 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:05:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:05:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:05:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:05:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:05:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:05:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:05:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:05:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:05:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:05:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:05:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:05:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:05:49.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:05:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:05:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:05:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:05:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:05:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:05:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:05:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:05:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:05:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:05:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:06:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:06:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:06:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:06:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:06:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:06:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:06:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:06:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:06:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:06:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:06:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:06:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:06:39.704 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:06:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:06:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:06:39.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:06:39.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:06:39.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:06:39.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:06:39.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:06:39.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:06:39.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:06:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:06:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:06:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:06:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:06:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:06:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:06:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:06:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:06:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:06:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:06:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:06:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:06:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:06:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:07:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:07:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:07:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:07:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:07:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:07:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:07:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:07:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:07:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:07:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:07:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:07:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:07:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:07:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:07:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:07:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:07:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:07:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:07:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:07:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:07:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:07:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:07:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:07:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:07:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:07:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:07:49.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:07:49.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:07:49.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:07:49.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:07:49.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:07:49.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:07:49.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:07:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:07:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:07:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:08:09.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:08:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:08:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:08:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:08:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:08:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:08:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:08:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:08:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:08:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:08:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:08:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:08:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:08:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:08:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:08:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:08:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:08:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:08:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:08:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:08:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:08:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:08:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:08:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:08:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:08:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:08:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:08:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:08:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:08:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:08:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:08:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:08:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:08:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:08:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:08:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:09:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:09:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:09:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:09:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:09:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:09:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:09:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:09:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:09:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:09:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:09:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:09:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:09:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:09:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:09:39.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:09:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:09:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:09:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:09:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:09:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:09:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:09:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:09:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:09:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:09:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:09:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:09:49.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:09:49.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:09:49.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:09:49.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:09:49.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:09:49.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:09:49.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:09:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:09:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:09:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:10:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:10:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:10:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:10:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:10:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:10:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:10:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:10:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:10:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:10:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:10:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:10:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:10:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:10:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:10:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:10:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:10:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:10:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:10:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:10:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:10:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:10:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:10:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:10:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:10:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:10:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:10:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:10:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:10:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:10:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:10:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:10:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:10:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:10:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:10:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:10:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:11:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:11:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:11:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:11:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:11:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:11:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:11:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:11:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:11:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:11:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:11:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:11:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:11:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:11:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:11:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:11:39.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:11:39.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:11:39.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:11:39.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:11:39.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:11:39.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:11:39.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:11:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:11:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.053 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:11:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:11:49.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:11:49.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:11:49.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:11:49.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:11:49.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:11:49.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:11:49.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:11:49.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:11:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:11:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.091 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:11:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:12:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:12:09.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:12:09.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:12:09.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:12:09.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:12:09.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:12:09.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:12:09.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:12:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:12:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:12:09.840 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:12:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:12:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:12:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:12:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:12:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:12:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:12:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:12:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:12:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:12:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:12:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:12:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:12:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:12:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:12:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:12:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:12:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:12:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:12:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:12:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:12:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:12:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:12:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:12:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:12:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:13:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:13:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:13:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:13:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:13:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:13:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:13:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:13:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:13:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:13:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:13:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:13:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:13:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:13:39.715 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:13:39.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:13:39.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:13:39.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:13:39.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:13:39.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:13:39.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:13:39.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:13:39.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:13:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:13:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:13:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:13:49.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:13:49.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:13:49.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:13:49.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:13:49.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:13:49.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:13:49.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:13:49.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:13:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:13:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:13:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:14:09.803 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:14:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:14:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:14:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:14:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:14:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:14:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:14:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:14:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:14:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:14:09.838 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:14:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:14:39.711 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:14:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:14:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:14:39.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:14:39.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:14:39.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:14:39.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:14:39.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:14:39.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:14:39.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:14:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:14:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.066 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:14:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:14:49.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:14:49.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:14:49.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:14:49.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:14:49.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:14:49.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:14:49.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:14:49.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:14:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:14:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:14:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:15:09.826 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:15:09.862 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:15:09.862 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:15:09.862 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:15:09.862 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:15:09.862 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:15:09.862 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:15:09.862 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:15:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:15:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.145 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:15:09.865 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:15:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:15:39.708 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:15:39.715 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:15:39.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:15:39.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:15:39.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:15:39.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:15:39.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:15:39.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:15:39.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:15:39.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:15:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:15:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:15:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:15:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:15:49.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:15:49.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:15:49.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:15:49.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:15:49.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:15:49.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:15:49.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:15:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:15:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:15:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:16:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:16:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:16:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:16:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:16:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:16:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:16:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:16:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:16:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:16:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:16:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:16:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:16:39.704 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:16:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:16:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:16:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:16:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:16:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:16:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:16:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:16:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:16:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:16:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:16:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:16:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:16:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:16:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:16:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:16:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:16:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:16:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:16:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:16:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:16:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:16:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:16:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:17:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:17:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:17:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:17:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:17:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:17:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:17:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:17:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:17:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:17:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:17:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:17:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:17:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:17:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:17:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:17:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:17:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:17:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:17:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:17:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:17:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:17:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:17:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:17:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:17:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:17:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:17:49.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:17:49.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:17:49.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:17:49.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:17:49.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:17:49.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:17:49.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:17:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:17:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:17:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:18:09.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:18:09.834 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:18:09.834 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:18:09.834 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:18:09.834 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:18:09.834 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:18:09.834 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:18:09.834 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:18:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:18:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:18:09.837 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:18:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:18:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:18:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:18:39.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:18:39.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:18:39.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:18:39.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:18:39.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:18:39.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:18:39.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:18:39.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:18:39.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:18:39.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.055 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:18:39.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:18:49.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:18:49.801 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:18:49.801 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:18:49.801 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:18:49.801 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:18:49.801 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:18:49.801 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:18:49.801 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:18:49.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:18:49.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.092 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:18:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:19:09.804 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:19:09.838 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:19:09.838 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:19:09.838 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:19:09.838 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:19:09.838 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:19:09.838 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:19:09.838 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:19:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:19:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:19:09.842 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:19:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:19:39.705 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:19:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:19:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:19:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:19:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:19:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:19:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:19:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:19:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:19:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:19:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:19:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:19:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:19:49.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:19:49.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:19:49.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:19:49.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:19:49.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:19:49.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:19:49.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:19:49.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:19:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:19:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:19:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:20:09.799 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:20:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:20:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:20:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:20:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:20:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:20:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:20:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:20:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:20:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.127 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:20:09.839 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:20:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:20:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:20:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:20:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:20:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:20:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:20:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:20:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:20:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:20:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:20:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:20:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:20:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:20:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:20:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:20:49.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:20:49.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:20:49.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:20:49.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:20:49.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:20:49.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:20:49.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:20:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:20:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:20:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:21:09.781 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:21:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:21:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:21:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:21:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:21:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:21:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:21:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:21:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:21:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:21:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:21:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:21:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:21:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:21:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:21:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:21:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:21:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:21:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:21:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:21:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:21:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:21:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:21:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:21:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:21:49.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:21:49.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:21:49.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:21:49.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:21:49.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:21:49.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:21:49.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:21:49.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:21:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:21:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:21:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:22:09.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:22:09.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:22:09.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:22:09.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:22:09.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:22:09.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:22:09.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:22:09.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:22:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:22:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:22:09.818 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:22:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:22:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:22:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:22:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:22:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:22:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:22:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:22:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:22:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:22:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:22:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:22:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:22:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:22:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:22:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:22:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:22:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:22:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:22:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:22:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:22:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:22:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:22:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:22:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:22:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:23:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:23:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:23:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:23:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:23:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:23:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:23:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:23:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:23:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:23:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:23:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:23:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:23:39.697 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:23:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:23:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:23:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:23:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:23:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:23:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:23:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:23:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:23:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:23:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:23:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:23:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:23:49.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:23:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:23:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:23:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:23:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:23:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:23:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:23:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:23:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:23:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:23:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:24:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:24:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:24:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:24:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:24:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:24:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:24:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:24:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:24:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:24:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:24:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:24:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:24:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:24:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:24:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:24:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:24:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:24:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:24:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:24:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:24:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:24:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:24:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:24:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:24:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:24:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:24:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:24:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:24:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:24:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:24:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:24:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:24:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:24:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:24:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:24:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:25:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:25:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:25:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:25:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:25:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:25:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:25:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:25:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:25:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:25:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:25:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:25:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:25:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:25:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:25:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:25:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:25:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:25:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:25:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:25:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:25:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:25:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:25:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:25:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:25:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:25:49.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:25:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:25:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:25:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:25:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:25:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:25:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:25:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:25:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:25:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:25:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:26:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:26:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:26:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:26:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:26:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:26:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:26:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:26:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:26:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:26:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:26:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:26:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:26:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:26:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:26:39.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:26:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:26:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:26:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:26:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:26:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:26:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:26:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:26:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:26:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:26:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:26:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:26:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:26:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:26:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:26:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:26:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:26:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:26:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:26:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:26:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:26:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:27:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:27:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:27:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:27:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:27:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:27:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:27:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:27:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:27:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:27:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:27:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:27:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:27:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:27:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:27:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:27:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:27:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:27:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:27:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:27:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:27:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:27:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:27:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:27:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:27:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:27:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:27:49.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:27:49.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:27:49.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:27:49.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:27:49.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:27:49.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:27:49.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:27:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:27:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:27:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:28:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:28:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:28:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:28:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:28:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:28:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:28:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:28:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:28:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:28:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:28:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:28:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:28:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:28:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:28:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:28:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:28:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:28:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:28:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:28:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:28:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:28:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:28:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:28:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:28:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:28:49.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:28:49.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:28:49.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:28:49.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:28:49.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:28:49.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:28:49.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:28:49.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:28:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:28:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:28:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:29:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:29:09.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:29:09.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:29:09.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:29:09.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:29:09.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:29:09.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:29:09.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:29:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:29:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:29:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:29:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:29:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:29:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:29:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:29:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:29:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:29:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:29:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:29:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:29:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:29:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:29:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:29:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:29:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:29:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:29:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:29:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:29:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:29:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:29:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:29:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:29:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:29:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:29:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:29:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:30:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:30:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:30:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:30:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:30:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:30:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:30:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:30:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:30:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:30:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:30:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:30:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:30:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:30:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:30:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:30:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:30:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:30:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:30:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:30:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:30:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:30:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:30:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:30:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:30:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:30:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:30:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:30:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:30:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:30:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:30:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:30:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:30:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:30:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:30:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:30:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:31:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:31:09.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:31:09.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:31:09.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:31:09.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:31:09.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:31:09.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:31:09.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:31:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:31:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:31:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:31:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:31:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:31:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:31:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:31:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:31:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:31:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:31:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:31:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:31:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:31:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:31:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:31:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.051 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:31:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:31:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:31:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:31:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:31:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:31:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:31:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:31:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:31:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:31:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:31:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:31:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:32:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:32:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:32:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:32:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:32:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:32:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:32:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:32:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:32:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:32:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:32:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:32:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:32:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:32:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:32:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:32:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:32:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:32:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:32:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:32:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:32:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:32:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:32:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:32:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:32:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:32:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:32:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:32:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:32:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:32:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:32:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:32:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:32:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:32:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:32:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:32:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:33:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:33:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:33:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:33:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:33:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:33:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:33:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:33:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:33:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:33:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:33:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:33:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:33:39.684 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:33:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:33:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:33:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:33:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:33:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:33:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:33:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:33:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:33:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:33:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:33:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:33:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:33:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:33:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:33:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:33:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:33:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:33:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:33:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:33:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:33:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:33:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:33:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:34:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:34:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:34:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:34:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:34:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:34:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:34:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:34:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:34:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:34:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:34:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:34:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:34:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:35:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:35:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:35:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:35:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:35:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:35:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:35:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:35:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:35:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:35:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:35:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:35:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:35:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:35:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:35:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:35:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:35:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:35:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:35:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:35:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:35:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:35:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:35:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:35:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:35:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:36:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:36:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:36:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:36:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:36:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:36:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:36:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:36:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:36:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:36:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:36:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:36:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:36:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:36:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:36:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:36:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:36:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:36:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:36:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:36:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:36:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:36:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:36:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:36:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:36:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:36:49.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:36:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:36:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:36:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:36:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:36:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:36:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:36:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:36:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:36:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:36:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:37:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:37:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:37:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:37:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:37:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:37:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:37:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:37:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:37:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:37:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:37:09.805 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:37:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:37:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:37:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:37:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:37:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:37:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:37:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:37:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:37:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:37:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:37:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:37:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:37:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:37:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:37:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:37:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:37:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:37:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:37:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:37:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:37:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:37:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:37:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:37:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:37:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:38:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:38:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:38:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:38:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:38:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:38:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:38:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:38:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:38:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:38:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:38:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:38:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:38:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:38:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:38:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:38:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:38:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:38:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:38:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:38:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:38:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:38:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:38:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:38:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:38:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:38:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:38:49.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:38:49.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:38:49.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:38:49.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:38:49.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:38:49.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:38:49.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:38:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:38:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:38:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:39:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:39:09.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:39:09.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:39:09.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:39:09.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:39:09.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:39:09.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:39:09.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:39:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:39:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:39:09.799 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:39:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:39:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:39:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:39:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:39:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:39:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:39:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:39:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:39:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:39:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:39:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:39:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:39:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:39:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:39:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:39:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:39:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:39:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:39:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:39:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:39:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:39:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:39:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:39:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:39:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:40:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:40:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:40:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:40:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:40:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:40:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:40:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:40:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:40:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:40:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:40:09.789 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:40:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:40:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:40:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:40:39.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:40:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:40:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:40:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:40:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:40:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:40:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:40:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:40:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:40:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:40:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:40:49.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:40:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:40:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:40:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:40:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:40:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:40:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:40:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:40:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:40:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:40:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:41:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:41:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:41:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:41:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:41:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:41:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:41:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:41:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:41:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:41:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:41:09.789 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:41:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:41:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:41:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:41:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:41:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:41:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:41:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:41:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:41:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:41:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:41:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:41:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:41:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:41:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:41:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:41:49.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:41:49.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:41:49.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:41:49.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:41:49.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:41:49.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:41:49.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:41:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:41:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:41:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:42:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:42:09.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:42:09.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:42:09.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:42:09.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:42:09.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:42:09.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:42:09.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:42:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:42:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:42:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:42:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:42:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:42:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:42:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:42:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:42:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:42:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:42:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:42:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:42:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:42:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:42:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:42:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:42:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:42:49.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:42:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:42:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:42:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:42:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:42:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:42:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:42:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:42:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:42:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:42:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:43:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:43:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:43:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:43:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:43:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:43:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:43:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:43:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:43:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:43:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:43:09.799 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:43:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:43:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:43:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:43:39.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:43:39.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:43:39.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:43:39.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:43:39.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:43:39.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:43:39.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:43:39.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:43:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:43:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:43:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:43:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:43:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:43:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:43:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:43:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:43:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:43:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:43:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:43:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:43:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:43:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:44:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:44:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:44:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:44:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:44:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:44:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:44:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:44:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:44:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:44:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:44:09.789 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:44:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:44:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:44:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:44:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:44:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:44:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:44:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:44:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:44:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:44:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:44:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:44:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:44:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:44:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:44:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:44:49.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:44:49.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:44:49.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:44:49.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:44:49.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:44:49.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:44:49.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:44:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:44:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:44:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:45:09.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:45:09.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:45:09.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:45:09.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:45:09.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:45:09.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:45:09.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:45:09.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:45:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:45:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:45:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:45:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:45:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:45:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:45:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:45:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:45:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:45:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:45:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:45:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:45:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:45:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:45:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:45:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:45:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:45:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:45:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:45:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:45:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:45:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:45:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:45:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:45:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:45:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:45:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:45:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:46:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:46:09.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:46:09.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:46:09.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:46:09.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:46:09.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:46:09.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:46:09.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:46:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:46:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:46:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:46:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:46:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:46:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:46:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:46:39.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:46:39.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:46:39.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:46:39.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:46:39.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:46:39.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:46:39.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:46:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:46:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:46:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:46:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:46:49.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:46:49.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:46:49.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:46:49.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:46:49.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:46:49.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:46:49.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:46:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:46:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:46:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:47:09.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:47:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:47:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:47:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:47:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:47:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:47:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:47:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:47:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:47:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:47:09.799 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:47:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:47:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:47:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:47:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:47:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:47:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:47:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:47:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:47:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:47:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:47:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:47:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:47:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.053 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:47:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:47:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:47:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:47:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:47:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:47:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:47:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:47:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:47:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:47:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:47:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.089 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:47:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:48:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:48:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:48:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:48:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:48:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:48:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:48:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:48:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:48:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:48:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:48:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:48:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:48:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:48:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:48:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:48:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:48:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:48:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:48:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:48:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:48:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:48:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:48:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:48:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:48:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:48:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:48:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:48:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:48:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:48:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:48:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:48:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:48:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:48:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:48:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:48:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:49:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:49:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:49:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:49:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:49:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:49:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:49:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:49:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:49:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:49:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:49:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:49:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:49:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:49:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:49:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:49:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:49:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:49:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:49:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:49:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:49:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:49:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:49:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:49:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:49:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:49:49.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:49:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:49:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:49:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:49:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:49:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:49:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:49:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:49:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:49:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:49:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:50:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:50:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:50:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:50:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:50:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:50:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:50:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:50:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:50:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:50:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:50:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:50:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:50:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:50:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:50:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:50:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:50:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:50:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:50:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:50:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:50:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:50:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:50:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:50:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:50:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:50:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:50:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:50:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:50:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:50:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:50:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:50:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:50:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:50:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:50:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:50:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:51:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:51:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:51:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:51:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:51:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:51:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:51:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:51:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:51:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:51:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:51:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:51:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:51:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:51:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:51:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:51:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:51:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:51:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:51:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:51:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:51:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:51:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:51:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:51:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:51:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:51:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:51:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:51:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:51:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:51:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:51:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:51:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:51:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:51:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:51:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:51:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:52:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:52:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:52:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:52:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:52:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:52:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:52:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:52:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:52:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:52:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:52:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:52:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:52:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:52:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:52:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:52:39.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:52:39.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:52:39.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:52:39.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:52:39.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:52:39.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:52:39.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:52:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:52:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:52:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:52:49.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:52:49.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:52:49.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:52:49.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:52:49.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:52:49.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:52:49.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:52:49.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:52:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:52:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:52:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:53:09.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:53:09.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:53:09.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:53:09.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:53:09.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:53:09.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:53:09.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:53:09.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:53:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:53:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:53:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:53:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:53:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:53:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:53:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:53:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:53:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:53:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:53:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:53:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:53:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:53:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:53:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:53:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:53:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:53:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:53:49.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:53:49.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:53:49.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:53:49.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:53:49.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:53:49.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:53:49.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:53:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:53:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:53:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:54:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:54:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:54:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:54:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:54:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:54:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:54:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:54:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:54:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:54:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:54:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:54:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:54:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:54:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:54:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:54:39.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:54:39.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:54:39.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:54:39.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:54:39.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:54:39.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:54:39.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:54:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:54:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:54:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:54:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:54:49.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:54:49.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:54:49.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:54:49.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:54:49.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:54:49.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:54:49.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:54:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:54:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:54:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:55:09.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:55:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:55:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:55:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:55:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:55:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:55:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:55:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:55:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:55:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:55:09.799 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:55:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:55:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:55:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:55:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:55:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:55:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:55:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:55:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:55:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:55:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:55:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:55:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:55:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:55:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:55:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:55:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:55:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:55:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:55:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:55:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:55:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:55:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:55:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:55:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:55:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:56:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:56:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:56:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:56:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:56:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:56:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:56:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:56:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:56:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:56:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:56:09.794 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:56:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:56:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:56:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:56:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:56:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:56:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:56:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:56:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:56:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:56:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:56:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:56:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:56:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:56:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:56:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:56:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:56:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:56:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:56:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:56:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:56:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:56:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:56:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:56:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:56:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:57:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:57:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:57:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:57:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:57:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:57:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:57:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:57:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:57:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:57:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:57:09.791 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:57:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:57:39.663 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:57:39.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:57:39.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:57:39.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:57:39.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:57:39.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:57:39.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:57:39.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:57:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:57:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:57:39.700 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:57:49.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:57:49.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:57:49.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:57:49.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:57:49.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:57:49.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:57:49.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:57:49.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:57:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:57:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:57:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:58:09.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:58:09.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:58:09.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:58:09.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:58:09.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:58:09.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:58:09.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:58:09.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:58:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:58:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:58:09.782 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:58:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:58:39.665 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:58:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:58:39.672 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:58:39.708 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:58:39.708 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:58:39.708 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:58:39.708 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:58:39.708 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:58:39.708 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:58:39.708 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:58:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:58:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:58:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:58:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:58:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:58:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:58:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:58:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:58:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:58:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:58:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:58:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:58:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.096 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:58:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:59:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:59:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:59:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:59:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:59:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:59:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:59:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:59:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:59:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:59:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.138 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 22:59:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-24 22:59:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:59:39.664 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 22:59:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 22:59:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:59:39.707 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:59:39.707 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:59:39.707 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:59:39.707 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:59:39.707 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:59:39.707 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:59:39.707 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:59:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:59:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:59:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 22:59:49.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 22:59:49.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 22:59:49.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 22:59:49.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 22:59:49.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 22:59:49.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 22:59:49.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 22:59:49.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 22:59:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 22:59:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 22:59:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:00:09.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:00:09.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:00:09.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:00:09.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:00:09.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:00:09.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:00:09.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:00:09.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:00:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:00:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:00:09.788 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:00:39.660 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:00:39.661 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:00:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:00:39.671 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:00:39.706 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:00:39.706 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:00:39.706 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:00:39.706 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:00:39.706 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:00:39.706 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:00:39.706 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:00:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:00:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:00:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:00:49.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:00:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:00:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:00:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:00:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:00:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:00:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:00:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:00:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:00:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:00:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:01:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:01:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:01:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:01:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:01:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:01:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:01:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:01:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:01:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:01:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:01:09.788 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:01:39.660 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:01:39.661 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:01:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:01:39.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:01:39.704 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:01:39.704 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:01:39.704 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:01:39.704 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:01:39.704 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:01:39.704 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:01:39.704 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:01:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:01:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:01:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:01:49.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:01:49.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:01:49.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:01:49.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:01:49.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:01:49.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:01:49.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:01:49.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:01:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:01:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:01:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:02:09.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:02:09.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:02:09.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:02:09.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:02:09.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:02:09.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:02:09.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:02:09.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:02:09.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:02:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:02:09.781 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:02:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:02:39.664 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:02:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:02:39.672 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:02:39.707 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:02:39.707 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:02:39.707 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:02:39.707 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:02:39.707 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:02:39.707 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:02:39.707 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:02:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:02:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:02:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:02:49.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:02:49.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:02:49.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:02:49.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:02:49.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:02:49.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:02:49.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:02:49.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:02:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:02:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:02:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:03:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:03:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:03:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:03:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:03:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:03:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:03:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:03:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:03:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:03:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:03:09.789 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:03:39.661 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:03:39.662 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:03:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:03:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:03:39.708 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:03:39.708 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:03:39.708 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:03:39.708 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:03:39.708 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:03:39.708 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:03:39.708 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:03:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:03:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:03:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:03:49.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:03:49.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:03:49.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:03:49.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:03:49.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:03:49.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:03:49.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:03:49.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:03:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:03:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:03:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:04:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:04:09.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:04:09.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:04:09.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:04:09.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:04:09.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:04:09.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:04:09.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:04:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:04:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:04:09.785 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:04:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:04:39.658 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:04:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:04:39.667 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:04:39.702 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:04:39.702 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:04:39.702 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:04:39.702 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:04:39.702 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:04:39.702 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:04:39.702 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:04:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:04:39.704 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:04:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:04:49.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:04:49.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:04:49.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:04:49.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:04:49.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:04:49.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:04:49.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:04:49.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:04:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:04:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:04:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:05:09.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:05:09.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:05:09.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:05:09.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:05:09.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:05:09.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:05:09.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:05:09.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:05:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:05:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:05:09.784 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:05:39.657 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:05:39.658 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:05:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:05:39.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:05:39.698 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:05:39.698 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:05:39.698 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:05:39.698 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:05:39.698 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:05:39.698 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:05:39.698 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:05:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:05:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:05:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:05:49.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:05:49.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:05:49.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:05:49.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:05:49.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:05:49.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:05:49.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:05:49.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:05:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:05:49.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:05:49.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:06:09.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:06:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:06:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:06:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:06:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:06:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:06:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:06:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:06:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:06:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:06:09.795 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:06:39.656 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:06:39.657 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:06:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:06:39.665 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:06:39.699 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:06:39.699 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:06:39.699 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:06:39.699 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:06:39.699 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:06:39.699 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:06:39.699 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:06:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:06:39.701 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:06:39.702 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:06:49.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:06:49.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:06:49.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:06:49.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:06:49.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:06:49.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:06:49.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:06:49.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:06:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:06:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:06:49.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:07:09.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:07:09.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:07:09.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:07:09.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:07:09.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:07:09.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:07:09.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:07:09.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:07:09.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:07:09.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:07:09.779 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:07:39.661 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:07:39.662 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:07:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:07:39.669 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:07:39.705 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:07:39.705 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:07:39.705 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:07:39.705 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:07:39.705 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:07:39.705 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:07:39.705 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:07:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:07:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:07:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:07:49.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:07:49.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:07:49.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:07:49.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:07:49.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:07:49.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:07:49.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:07:49.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:07:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:07:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:07:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:08:09.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:08:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:08:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:08:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:08:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:08:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:08:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:08:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:08:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:08:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:08:09.791 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:08:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:08:39.663 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:08:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:08:39.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:08:39.704 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:08:39.704 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:08:39.704 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:08:39.704 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:08:39.704 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:08:39.704 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:08:39.704 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:08:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:08:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:08:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:08:49.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:08:49.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:08:49.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:08:49.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:08:49.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:08:49.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:08:49.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:08:49.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:08:49.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:08:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:08:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:09:09.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:09:09.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:09:09.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:09:09.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:09:09.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:09:09.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:09:09.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:09:09.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:09:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:09:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:09:09.781 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:09:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:09:39.663 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:09:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:09:39.674 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:09:39.708 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:09:39.708 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:09:39.708 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:09:39.708 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:09:39.708 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:09:39.708 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:09:39.708 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:09:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:09:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:09:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:09:49.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:09:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:09:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:09:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:09:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:09:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:09:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:09:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:09:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:09:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:09:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:10:09.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:10:09.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:10:09.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:10:09.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:10:09.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:10:09.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:10:09.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:10:09.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:10:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:10:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:10:09.791 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:10:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:10:39.664 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:10:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:10:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:10:39.708 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:10:39.708 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:10:39.708 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:10:39.708 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:10:39.708 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:10:39.708 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:10:39.708 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:10:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:10:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:10:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:10:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:10:49.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:10:49.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:10:49.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:10:49.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:10:49.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:10:49.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:10:49.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:10:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:10:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:10:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:11:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:11:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:11:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:11:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:11:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:11:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:11:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:11:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:11:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:11:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:11:09.801 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:11:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:11:39.664 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:11:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:11:39.672 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:11:39.704 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:11:39.704 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:11:39.704 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:11:39.704 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:11:39.704 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:11:39.704 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:11:39.704 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:11:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:11:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:11:39.707 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:11:49.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:11:49.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:11:49.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:11:49.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:11:49.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:11:49.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:11:49.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:11:49.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:11:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:11:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:11:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:12:09.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:12:09.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:12:09.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:12:09.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:12:09.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:12:09.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:12:09.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:12:09.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:12:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:12:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:12:09.782 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:12:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:12:39.665 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:12:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:12:39.672 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:12:39.706 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:12:39.706 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:12:39.706 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:12:39.706 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:12:39.706 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:12:39.706 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:12:39.706 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:12:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:12:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:12:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:12:49.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:12:49.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:12:49.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:12:49.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:12:49.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:12:49.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:12:49.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:12:49.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:12:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:12:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:12:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:13:09.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:13:09.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:13:09.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:13:09.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:13:09.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:13:09.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:13:09.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:13:09.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:13:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:13:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:13:09.785 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:13:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:13:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:13:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:13:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:13:39.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:13:39.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:13:39.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:13:39.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:13:39.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:13:39.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:13:39.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:13:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:13:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:13:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:13:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:13:49.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:13:49.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:13:49.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:13:49.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:13:49.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:13:49.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:13:49.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:13:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:13:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:13:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:14:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:14:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:14:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:14:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:14:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:14:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:14:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:14:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:14:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:14:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:14:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:14:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:14:39.665 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:14:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:14:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:14:39.706 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:14:39.706 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:14:39.706 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:14:39.706 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:14:39.706 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:14:39.706 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:14:39.706 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:14:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:14:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:14:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:14:49.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:14:49.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:14:49.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:14:49.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:14:49.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:14:49.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:14:49.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:14:49.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:14:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:14:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:14:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:15:09.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:15:09.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:15:09.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:15:09.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:15:09.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:15:09.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:15:09.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:15:09.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:15:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:15:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:15:09.785 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:15:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:15:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:15:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:15:39.674 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:15:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:15:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:15:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:15:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:15:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:15:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:15:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:15:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:15:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:15:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:15:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:15:49.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:15:49.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:15:49.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:15:49.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:15:49.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:15:49.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:15:49.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:15:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:15:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:15:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:16:09.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:16:09.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:16:09.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:16:09.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:16:09.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:16:09.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:16:09.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:16:09.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:16:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:16:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:16:09.785 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:16:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:16:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:16:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:16:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:16:39.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:16:39.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:16:39.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:16:39.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:16:39.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:16:39.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:16:39.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:16:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:16:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:16:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:16:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:16:49.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:16:49.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:16:49.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:16:49.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:16:49.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:16:49.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:16:49.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:16:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:16:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:16:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:17:09.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:17:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:17:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:17:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:17:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:17:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:17:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:17:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:17:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:17:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:17:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:17:39.665 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:17:39.666 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:17:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:17:39.674 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:17:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:17:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:17:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:17:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:17:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:17:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:17:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:17:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:17:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:17:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:17:49.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:17:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:17:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:17:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:17:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:17:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:17:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:17:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:17:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:17:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:17:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:18:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:18:09.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:18:09.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:18:09.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:18:09.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:18:09.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:18:09.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:18:09.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:18:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:18:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:18:09.787 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:18:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:18:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:18:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:18:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:18:39.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:18:39.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:18:39.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:18:39.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:18:39.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:18:39.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:18:39.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:18:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:18:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:18:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:18:49.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:18:49.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:18:49.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:18:49.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:18:49.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:18:49.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:18:49.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:18:49.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:18:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:18:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:18:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:19:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:19:09.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:19:09.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:19:09.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:19:09.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:19:09.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:19:09.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:19:09.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:19:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:19:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:19:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:19:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:19:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:19:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:19:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:19:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:19:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:19:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:19:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:19:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:19:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:19:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:19:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:19:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:19:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:19:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:19:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:19:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:19:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:19:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:19:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:19:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:19:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:19:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:19:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:19:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:20:09.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:20:09.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:20:09.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:20:09.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:20:09.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:20:09.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:20:09.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:20:09.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:20:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:20:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:20:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:20:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:20:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:20:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:20:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:20:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:20:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:20:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:20:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:20:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:20:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:20:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:20:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:20:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:20:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:20:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:20:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:20:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:20:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:20:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:20:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:20:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:20:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:20:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:20:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:20:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:21:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:21:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:21:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:21:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:21:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:21:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:21:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:21:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:21:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:21:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:21:09.796 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:21:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:21:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:21:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:21:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:21:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:21:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:21:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:21:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:21:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:21:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:21:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:21:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:21:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:21:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:21:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:21:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:21:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:21:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:21:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:21:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:21:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:21:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:21:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:21:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:21:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:22:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:22:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:22:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:22:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:22:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:22:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:22:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:22:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:22:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:22:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:22:09.796 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:22:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:22:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:22:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:22:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:22:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:22:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:22:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:22:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:22:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:22:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:22:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:22:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:22:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:22:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:22:49.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:22:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:22:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:22:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:22:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:22:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:22:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:22:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:22:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:22:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:22:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:23:09.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:23:09.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:23:09.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:23:09.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:23:09.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:23:09.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:23:09.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:23:09.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:23:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:23:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:23:09.796 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:23:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:23:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:23:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:23:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:23:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:23:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:23:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:23:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:23:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:23:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:23:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:23:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:23:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:23:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:23:49.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:23:49.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:23:49.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:23:49.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:23:49.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:23:49.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:23:49.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:23:49.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:23:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:23:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:23:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:24:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:24:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:24:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:24:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:24:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:24:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:24:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:24:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:24:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:24:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:24:09.796 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:24:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:24:39.668 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:24:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:24:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:24:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:24:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:24:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:24:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:24:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:24:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:24:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:24:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:24:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:24:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:24:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:24:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:24:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:24:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:24:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:24:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:24:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:24:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:24:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:24:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:24:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:25:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:25:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:25:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:25:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:25:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:25:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:25:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:25:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:25:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:25:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:25:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:25:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:25:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:25:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:25:39.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:25:39.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:25:39.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:25:39.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:25:39.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:25:39.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:25:39.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:25:39.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:25:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:25:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:25:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:25:49.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:25:49.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:25:49.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:25:49.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:25:49.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:25:49.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:25:49.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:25:49.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:25:49.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:25:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:25:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:26:09.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:26:09.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:26:09.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:26:09.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:26:09.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:26:09.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:26:09.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:26:09.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:26:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:26:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:26:09.784 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:26:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:26:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:26:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:26:39.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:26:39.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:26:39.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:26:39.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:26:39.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:26:39.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:26:39.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:26:39.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:26:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:26:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:26:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:26:49.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:26:49.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:26:49.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:26:49.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:26:49.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:26:49.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:26:49.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:26:49.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:26:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:26:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:26:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:27:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:27:09.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:27:09.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:27:09.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:27:09.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:27:09.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:27:09.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:27:09.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:27:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:27:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:27:09.791 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:27:39.663 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:27:39.663 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:27:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:27:39.671 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:27:39.706 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:27:39.706 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:27:39.706 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:27:39.706 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:27:39.706 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:27:39.706 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:27:39.706 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:27:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:27:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:27:39.708 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:27:49.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:27:49.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:27:49.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:27:49.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:27:49.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:27:49.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:27:49.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:27:49.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:27:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:27:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:27:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:28:09.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:28:09.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:28:09.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:28:09.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:28:09.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:28:09.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:28:09.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:28:09.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:28:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:28:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:28:09.782 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:28:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:28:39.665 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:28:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:28:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:28:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:28:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:28:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:28:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:28:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:28:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:28:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:28:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:28:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:28:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:28:49.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:28:49.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:28:49.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:28:49.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:28:49.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:28:49.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:28:49.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:28:49.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:28:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:28:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:28:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:29:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:29:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:29:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:29:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:29:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:29:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:29:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:29:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:29:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:29:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:29:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:29:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:29:39.667 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:29:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:29:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:29:39.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:29:39.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:29:39.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:29:39.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:29:39.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:29:39.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:29:39.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:29:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:29:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:29:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:29:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:29:49.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:29:49.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:29:49.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:29:49.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:29:49.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:29:49.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:29:49.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:29:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:29:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:29:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:30:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:30:09.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:30:09.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:30:09.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:30:09.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:30:09.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:30:09.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:30:09.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:30:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:30:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:30:09.794 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:30:39.667 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:30:39.668 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:30:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:30:39.678 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:30:39.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:30:39.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:30:39.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:30:39.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:30:39.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:30:39.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:30:39.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:30:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:30:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:30:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:30:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:30:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:30:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:30:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:30:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:30:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:30:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:30:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:30:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:30:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:30:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:31:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:31:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:31:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:31:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:31:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:31:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:31:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:31:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:31:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:31:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:31:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:31:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:31:39.663 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:31:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:31:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:31:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:31:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:31:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:31:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:31:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:31:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:31:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:31:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:31:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:31:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:31:49.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:31:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:31:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:31:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:31:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:31:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:31:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:31:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:31:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:31:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:31:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:32:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:32:09.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:32:09.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:32:09.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:32:09.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:32:09.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:32:09.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:32:09.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:32:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:32:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:32:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:32:39.662 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:32:39.663 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:32:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:32:39.672 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:32:39.707 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:32:39.707 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:32:39.707 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:32:39.707 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:32:39.707 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:32:39.707 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:32:39.707 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:32:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:32:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:32:39.711 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:32:49.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:32:49.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:32:49.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:32:49.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:32:49.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:32:49.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:32:49.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:32:49.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:32:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:32:49.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:32:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:33:09.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:33:09.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:33:09.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:33:09.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:33:09.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:33:09.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:33:09.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:33:09.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:33:09.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:33:09.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:33:09.784 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:33:39.666 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:33:39.666 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:33:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:33:39.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:33:39.710 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:33:39.710 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:33:39.710 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:33:39.710 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:33:39.710 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:33:39.710 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:33:39.710 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:33:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:33:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:33:39.713 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:33:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:33:49.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:33:49.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:33:49.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:33:49.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:33:49.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:33:49.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:33:49.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:33:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:33:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:33:49.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:34:09.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:34:09.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:34:09.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:34:09.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:34:09.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:34:09.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:34:09.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:34:09.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:34:09.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:34:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:34:09.786 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:34:39.668 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:34:39.669 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:34:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:34:39.676 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:34:39.707 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:34:39.707 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:34:39.707 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:34:39.707 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:34:39.707 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:34:39.707 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:34:39.707 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:34:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:34:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.032 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:34:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:34:49.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:34:49.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:34:49.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:34:49.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:34:49.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:34:49.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:34:49.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:34:49.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:34:49.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:34:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.068 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:34:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:35:09.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:35:09.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:35:09.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:35:09.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:35:09.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:35:09.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:35:09.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:35:09.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:35:09.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:35:09.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:35:09.782 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:35:39.664 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:35:39.665 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:35:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:35:39.673 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:35:39.707 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:35:39.707 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:35:39.707 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:35:39.707 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:35:39.707 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:35:39.707 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:35:39.707 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:35:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:35:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:35:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:35:49.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:35:49.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:35:49.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:35:49.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:35:49.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:35:49.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:35:49.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:35:49.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:35:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:35:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:35:49.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:36:09.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:36:09.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:36:09.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:36:09.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:36:09.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:36:09.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:36:09.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:36:09.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:36:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:36:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:36:09.787 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:36:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:36:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:36:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:36:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:36:39.711 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:36:39.711 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:36:39.711 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:36:39.711 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:36:39.711 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:36:39.711 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:36:39.711 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:36:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:36:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:36:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:36:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:36:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:36:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:36:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:36:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:36:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:36:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:36:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:36:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:36:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:36:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:37:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:37:09.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:37:09.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:37:09.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:37:09.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:37:09.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:37:09.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:37:09.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:37:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:37:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:37:09.792 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:37:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:37:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:37:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:37:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:37:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:37:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:37:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:37:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:37:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:37:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:37:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:37:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:37:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:37:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:37:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:37:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:37:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:37:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:37:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:37:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:37:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:37:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:37:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:37:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:37:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:38:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:38:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:38:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:38:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:38:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:38:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:38:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:38:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:38:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:38:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:38:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:38:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:38:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:38:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:38:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:38:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:38:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:38:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:38:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:38:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:38:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:38:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:38:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:38:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:38:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:38:49.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:38:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:38:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:38:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:38:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:38:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:38:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:38:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:38:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:38:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:38:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:39:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:39:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:39:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:39:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:39:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:39:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:39:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:39:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:39:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:39:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:39:09.792 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:39:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:39:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:39:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:39:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:39:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:39:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:39:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:39:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:39:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:39:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:39:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:39:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:39:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:39:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:39:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:39:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:39:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:39:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:39:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:39:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:39:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:39:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:39:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:39:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:39:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:40:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:40:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:40:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:40:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:40:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:40:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:40:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:40:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:40:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:40:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:40:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:40:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:40:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:41:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:41:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:41:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:41:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:41:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:41:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:41:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:41:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:41:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:41:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:41:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:41:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:41:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:41:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:41:49.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:41:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:41:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:41:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:41:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:41:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:41:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:41:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:41:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:41:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.089 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:41:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:42:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:42:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:42:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:42:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:42:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:42:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:42:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:42:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:42:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:42:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:42:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:42:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:42:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:42:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:42:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:42:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:42:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:42:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:42:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:42:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:42:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:42:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:42:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:42:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:42:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:42:49.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:42:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:42:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:42:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:42:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:42:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:42:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:42:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:42:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:42:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:42:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:43:09.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:43:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:43:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:43:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:43:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:43:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:43:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:43:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:43:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:43:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:43:09.796 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:43:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:43:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:43:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:43:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:43:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:43:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:43:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:43:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:43:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:43:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:43:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:43:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:43:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:43:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:43:49.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:43:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:43:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:43:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:43:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:43:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:43:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:43:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:43:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:43:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:43:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:44:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:44:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:44:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:44:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:44:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:44:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:44:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:44:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:44:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:44:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:44:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:44:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:44:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:44:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:44:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:44:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:44:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:44:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:44:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:44:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:44:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:44:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:44:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:44:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:44:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:44:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:44:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:44:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:44:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:44:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:44:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:44:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:44:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:44:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:44:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:44:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:45:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:45:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:45:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:45:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:45:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:45:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:45:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:45:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:45:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:45:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:45:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:45:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:45:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:45:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:45:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:45:39.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:45:39.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:45:39.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:45:39.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:45:39.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:45:39.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:45:39.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:45:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:45:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:45:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:45:49.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:45:49.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:45:49.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:45:49.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:45:49.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:45:49.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:45:49.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:45:49.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:45:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:45:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:45:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:46:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:46:09.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:46:09.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:46:09.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:46:09.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:46:09.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:46:09.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:46:09.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:46:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:46:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:46:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:46:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:46:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:46:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:46:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:46:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:46:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:46:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:46:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:46:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:46:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:46:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:46:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:46:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:46:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:46:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:46:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:46:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:46:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:46:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:46:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:46:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:46:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:46:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:46:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:46:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:47:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:47:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:47:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:47:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:47:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:47:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:47:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:47:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:47:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:47:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:47:09.792 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:47:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:47:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:47:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:47:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:47:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:47:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:47:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:47:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:47:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:47:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:47:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:47:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:47:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:47:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:47:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:47:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:47:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:47:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:47:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:47:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:47:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:47:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:47:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:47:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:47:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:48:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:48:09.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:48:09.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:48:09.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:48:09.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:48:09.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:48:09.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:48:09.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:48:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:48:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:48:09.795 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:48:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:48:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:48:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:48:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:48:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:48:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:48:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:48:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:48:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:48:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:48:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:48:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:48:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.057 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:48:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:48:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:48:49.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:48:49.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:48:49.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:48:49.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:48:49.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:48:49.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:48:49.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:48:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:48:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.095 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:48:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:49:09.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:49:09.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:49:09.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:49:09.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:49:09.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:49:09.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:49:09.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:49:09.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:49:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:49:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.132 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:49:09.819 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:49:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:49:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:49:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:49:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:49:39.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:49:39.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:49:39.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:49:39.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:49:39.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:49:39.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:49:39.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:49:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:49:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:49:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:49:49.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:49:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:49:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:49:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:49:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:49:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:49:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:49:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:49:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:49:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:49:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:50:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:50:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:50:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:50:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:50:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:50:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:50:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:50:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:50:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:50:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:50:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:50:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:50:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:50:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:50:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:50:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:50:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:50:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:50:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:50:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:50:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:50:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:50:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:50:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:50:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:50:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:50:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:50:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:50:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:50:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:50:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:50:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:50:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:50:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:50:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:50:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:51:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:51:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:51:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:51:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:51:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:51:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:51:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:51:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:51:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:51:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:51:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:51:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:51:39.675 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:51:39.707 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:51:39.707 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:51:39.707 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:51:39.707 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:51:39.707 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:51:39.707 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:51:39.707 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:51:39.709 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:51:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:51:39.710 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:51:49.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:51:49.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:51:49.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:51:49.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:51:49.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:51:49.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:51:49.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:51:49.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:51:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:51:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:51:49.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:52:09.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:52:09.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:52:09.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:52:09.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:52:09.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:52:09.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:52:09.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:52:09.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:52:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:52:09.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:52:09.787 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:52:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:52:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:52:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:52:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:52:39.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:52:39.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:52:39.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:52:39.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:52:39.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:52:39.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:52:39.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:52:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:52:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:52:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:52:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:52:49.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:52:49.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:52:49.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:52:49.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:52:49.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:52:49.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:52:49.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:52:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:52:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:52:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:53:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:53:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:53:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:53:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:53:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:53:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:53:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:53:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:53:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:53:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:53:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:53:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:53:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:53:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:53:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:53:39.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:53:39.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:53:39.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:53:39.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:53:39.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:53:39.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:53:39.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:53:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:53:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:53:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:53:49.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:53:49.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:53:49.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:53:49.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:53:49.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:53:49.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:53:49.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:53:49.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:53:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:53:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:53:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:54:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:54:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:54:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:54:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:54:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:54:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:54:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:54:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:54:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:54:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:54:09.802 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:54:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:54:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:54:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:54:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:54:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:54:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:54:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:54:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:54:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:54:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:54:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:54:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:54:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:54:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:54:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:54:49.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:54:49.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:54:49.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:54:49.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:54:49.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:54:49.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:54:49.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:54:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:54:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:54:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:55:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:55:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:55:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:55:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:55:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:55:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:55:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:55:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:55:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:55:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:55:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:55:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:55:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:55:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:55:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:55:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:55:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:55:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:55:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:55:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:55:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:55:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:55:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:55:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:55:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:55:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:55:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:55:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:55:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:55:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:55:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:55:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:55:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:55:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:55:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:55:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:56:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:56:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:56:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:56:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:56:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:56:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:56:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:56:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:56:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:56:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:56:09.805 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:56:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:56:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:56:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:56:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:56:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:56:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:56:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:56:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:56:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:56:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:56:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:56:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:56:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:56:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:56:49.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:56:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:56:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:56:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:56:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:56:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:56:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:56:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:56:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:56:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:56:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:57:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:57:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:57:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:57:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:57:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:57:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:57:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:57:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:57:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:57:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:57:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:57:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:57:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:57:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:57:39.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:57:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:57:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:57:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:57:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:57:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:57:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:57:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:57:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:57:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:57:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:57:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:57:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:57:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:57:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:57:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:57:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:57:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:57:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:57:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:57:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:57:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:58:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:58:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:58:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:58:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:58:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:58:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:58:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:58:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:58:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:58:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:58:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:58:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:58:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:58:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:58:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:58:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:58:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:58:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:58:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:58:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:58:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:58:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:58:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:58:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:58:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:58:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:58:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:58:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:58:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:58:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:58:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:58:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:58:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:58:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:58:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:58:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:59:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:59:09.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:59:09.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:59:09.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:59:09.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:59:09.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:59:09.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:59:09.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:59:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:59:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-24 23:59:09.794 139328 ERROR oslo_service.periodic_task 
>2018-06-24 23:59:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:59:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-24 23:59:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-24 23:59:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:59:39.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:59:39.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:59:39.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:59:39.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:59:39.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:59:39.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:59:39.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:59:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:59:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:59:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-24 23:59:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-24 23:59:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-24 23:59:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-24 23:59:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-24 23:59:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-24 23:59:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-24 23:59:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-24 23:59:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-24 23:59:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-24 23:59:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.095 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-24 23:59:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:00:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:00:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:00:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:00:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:00:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:00:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:00:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:00:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:00:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:00:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.133 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:00:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:00:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:00:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:00:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:00:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:00:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:00:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:00:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:00:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:00:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:00:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:00:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:00:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:00:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:00:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:00:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:00:49.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:00:49.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:00:49.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:00:49.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:00:49.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:00:49.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:00:49.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:00:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:00:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:00:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:01:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:01:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:01:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:01:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:01:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:01:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:01:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:01:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:01:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:01:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.127 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:01:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:01:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:01:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:01:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:01:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:01:39.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:01:39.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:01:39.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:01:39.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:01:39.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:01:39.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:01:39.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:01:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:01:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:01:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:01:49.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:01:49.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:01:49.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:01:49.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:01:49.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:01:49.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:01:49.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:01:49.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:01:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:01:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:01:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:02:09.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:02:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:02:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:02:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:02:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:02:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:02:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:02:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:02:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:02:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:02:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:02:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:02:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:02:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:02:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:02:39.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:02:39.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:02:39.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:02:39.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:02:39.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:02:39.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:02:39.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:02:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:02:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:02:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:02:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:02:49.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:02:49.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:02:49.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:02:49.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:02:49.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:02:49.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:02:49.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:02:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:02:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:02:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:03:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:03:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:03:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:03:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:03:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:03:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:03:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:03:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:03:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:03:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:03:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:03:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:03:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:03:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:03:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:03:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:03:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:03:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:03:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:03:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:03:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:03:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:03:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:03:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:03:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:03:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:03:49.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:03:49.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:03:49.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:03:49.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:03:49.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:03:49.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:03:49.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:03:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:03:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:03:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:04:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:04:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:04:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:04:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:04:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:04:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:04:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:04:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:04:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:04:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:04:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:04:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:04:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:04:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:04:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:04:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:04:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:04:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:04:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:04:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:04:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:04:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:04:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:04:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:04:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:04:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:04:49.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:04:49.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:04:49.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:04:49.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:04:49.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:04:49.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:04:49.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:04:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:04:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:04:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:05:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:05:09.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:05:09.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:05:09.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:05:09.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:05:09.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:05:09.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:05:09.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:05:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:05:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:05:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:05:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:05:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:05:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:05:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:05:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:05:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:05:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:05:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:05:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:05:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:05:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:05:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:05:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:05:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:05:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:05:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:05:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:05:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:05:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:05:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:05:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:05:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:05:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:05:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:05:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:06:09.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:06:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:06:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:06:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:06:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:06:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:06:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:06:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:06:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:06:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:06:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:06:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:06:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:06:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:06:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:06:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:06:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:06:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:06:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:06:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:06:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:06:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:06:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:06:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:06:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:06:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:06:49.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:06:49.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:06:49.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:06:49.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:06:49.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:06:49.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:06:49.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:06:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:06:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:06:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:07:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:07:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:07:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:07:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:07:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:07:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:07:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:07:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:07:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:07:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:07:09.796 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:07:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:07:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:07:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:07:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:07:39.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:07:39.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:07:39.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:07:39.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:07:39.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:07:39.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:07:39.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:07:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:07:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:07:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:07:49.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:07:49.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:07:49.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:07:49.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:07:49.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:07:49.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:07:49.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:07:49.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:07:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:07:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:07:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:08:09.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:08:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:08:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:08:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:08:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:08:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:08:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:08:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:08:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:08:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.128 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:08:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:08:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:08:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:08:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:08:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:08:39.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:08:39.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:08:39.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:08:39.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:08:39.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:08:39.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:08:39.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:08:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:08:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:08:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:08:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:08:49.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:08:49.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:08:49.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:08:49.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:08:49.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:08:49.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:08:49.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:08:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:08:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:08:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:09:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:09:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:09:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:09:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:09:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:09:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:09:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:09:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:09:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:09:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:09:09.798 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:09:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:09:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:09:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:09:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:09:39.720 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:09:39.720 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:09:39.720 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:09:39.720 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:09:39.720 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:09:39.720 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:09:39.720 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:09:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:09:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:09:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:09:49.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:09:49.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:09:49.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:09:49.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:09:49.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:09:49.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:09:49.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:09:49.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:09:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:09:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:09:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:10:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:10:09.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:10:09.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:10:09.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:10:09.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:10:09.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:10:09.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:10:09.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:10:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:10:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:10:09.799 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:10:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:10:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:10:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:10:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:10:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:10:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:10:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:10:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:10:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:10:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:10:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:10:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:10:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:10:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:10:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:10:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:10:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:10:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:10:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:10:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:10:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:10:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:10:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:10:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:10:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:11:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:11:09.818 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:11:09.818 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:11:09.818 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:11:09.818 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:11:09.818 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:11:09.818 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:11:09.818 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:11:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:11:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:11:09.821 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:11:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:11:39.684 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:11:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:11:39.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:11:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:11:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:11:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:11:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:11:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:11:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:11:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:11:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:11:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:11:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:11:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:11:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:11:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:11:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:11:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:11:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:11:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:11:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:11:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:11:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:11:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:12:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:12:09.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:12:09.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:12:09.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:12:09.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:12:09.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:12:09.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:12:09.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:12:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:12:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:12:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:12:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:12:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:12:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:12:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:12:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:12:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:12:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:12:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:12:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:12:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:12:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:12:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:12:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:12:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:12:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:12:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:12:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:12:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:12:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:12:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:12:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:12:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:12:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:12:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:12:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:13:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:13:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:13:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:13:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:13:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:13:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:13:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:13:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:13:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:13:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:13:09.814 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:13:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:13:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:13:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:13:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:13:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:13:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:13:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:13:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:13:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:13:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:13:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:13:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:13:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:13:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:13:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:13:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:13:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:13:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:13:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:13:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:13:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:13:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:13:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:13:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:13:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:14:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:14:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:14:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:14:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:14:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:14:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:14:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:14:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:14:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:14:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:14:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:14:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:14:39.686 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:14:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:14:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:14:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:14:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:14:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:14:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:14:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:14:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:14:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:14:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:14:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:14:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:14:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:14:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:14:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:14:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:14:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:14:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:14:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:14:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:14:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:14:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:14:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:15:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:15:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:15:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:15:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:15:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:15:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:15:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:15:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:15:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:15:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:15:09.802 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:15:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:15:39.684 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:15:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:15:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:15:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:15:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:15:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:15:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:15:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:15:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:15:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:15:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:15:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:15:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:15:49.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:15:49.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:15:49.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:15:49.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:15:49.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:15:49.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:15:49.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:15:49.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:15:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:15:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:15:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:16:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:16:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:16:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:16:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:16:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:16:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:16:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:16:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:16:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:16:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:16:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:16:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:16:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:16:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:16:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:16:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:16:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:16:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:16:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:16:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:16:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:16:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:16:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:16:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:16:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:16:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:16:49.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:16:49.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:16:49.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:16:49.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:16:49.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:16:49.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:16:49.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:16:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:16:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:16:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:17:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:17:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:17:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:17:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:17:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:17:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:17:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:17:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:17:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:17:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:17:09.808 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:17:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:17:39.691 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:17:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:17:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:17:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:17:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:17:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:17:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:17:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:17:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:17:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:17:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:17:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:17:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:17:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:17:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:17:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:17:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:17:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:17:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:17:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:17:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:17:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:17:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:17:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:18:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:18:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:18:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:18:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:18:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:18:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:18:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:18:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:18:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:18:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:18:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:18:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:18:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:18:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:18:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:18:39.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:18:39.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:18:39.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:18:39.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:18:39.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:18:39.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:18:39.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:18:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:18:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:18:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:18:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:18:49.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:18:49.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:18:49.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:18:49.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:18:49.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:18:49.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:18:49.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:18:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:18:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:18:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:19:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:19:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:19:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:19:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:19:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:19:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:19:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:19:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:19:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:19:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:19:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:19:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:19:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:19:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:19:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:19:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:19:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:19:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:19:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:19:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:19:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:19:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:19:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:19:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:19:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:19:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:19:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:19:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:19:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:19:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:19:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:19:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:19:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:19:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:19:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:19:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:20:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:20:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:20:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:20:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:20:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:20:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:20:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:20:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:20:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:20:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:20:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:20:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:20:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:20:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:20:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:20:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:20:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:20:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:20:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:20:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:20:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:20:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:20:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:20:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:20:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:20:49.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:20:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:20:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:20:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:20:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:20:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:20:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:20:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:20:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:20:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:20:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:21:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:21:09.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:21:09.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:21:09.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:21:09.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:21:09.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:21:09.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:21:09.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:21:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:21:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:21:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:21:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:21:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:21:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:21:39.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:21:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:21:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:21:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:21:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:21:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:21:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:21:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:21:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:21:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.053 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:21:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:21:49.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:21:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:21:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:21:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:21:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:21:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:21:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:21:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:21:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:21:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.09 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:21:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:22:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:22:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:22:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:22:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:22:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:22:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:22:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:22:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:22:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:22:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:22:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:22:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:22:39.691 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:22:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:22:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:22:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:22:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:22:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:22:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:22:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:22:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:22:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:22:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:22:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:22:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:22:49.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:22:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:22:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:22:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:22:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:22:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:22:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:22:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:22:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:22:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:22:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:23:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:23:09.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:23:09.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:23:09.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:23:09.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:23:09.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:23:09.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:23:09.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:23:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:23:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:23:09.814 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:23:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:23:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:23:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:23:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:23:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:23:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:23:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:23:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:23:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:23:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:23:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:23:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:23:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:23:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:23:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:23:49.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:23:49.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:23:49.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:23:49.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:23:49.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:23:49.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:23:49.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:23:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:23:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:23:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:24:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:24:09.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:24:09.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:24:09.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:24:09.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:24:09.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:24:09.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:24:09.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:24:09.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:24:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:24:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:24:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:24:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:24:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:24:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:24:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:24:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:24:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:24:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:24:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:24:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:24:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:24:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:24:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:24:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:24:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:24:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:24:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:24:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:24:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:24:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:24:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:24:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:24:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:24:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:24:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:25:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:25:09.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:25:09.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:25:09.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:25:09.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:25:09.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:25:09.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:25:09.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:25:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:25:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:25:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:25:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:25:39.692 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:25:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:25:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:25:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:25:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:25:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:25:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:25:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:25:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:25:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:25:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:25:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:25:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:25:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:25:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:25:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:25:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:25:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:25:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:25:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:25:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:25:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:25:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:25:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:26:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:26:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:26:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:26:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:26:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:26:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:26:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:26:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:26:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:26:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:26:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:26:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:26:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:26:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:26:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:26:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:26:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:26:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:26:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:26:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:26:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:26:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:26:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:26:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:26:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:26:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:26:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:26:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:26:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:26:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:26:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:26:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:26:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:26:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:26:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:26:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:27:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:27:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:27:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:27:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:27:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:27:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:27:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:27:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:27:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:27:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:27:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:27:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:27:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:27:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:27:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:27:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:27:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:27:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:27:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:27:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:27:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:27:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:27:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:27:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:27:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:27:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:27:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:27:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:27:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:27:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:27:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:27:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:27:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:27:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:27:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:27:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:28:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:28:09.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:28:09.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:28:09.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:28:09.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:28:09.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:28:09.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:28:09.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:28:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:28:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:28:09.814 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:28:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:28:39.686 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:28:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:28:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:28:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:28:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:28:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:28:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:28:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:28:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:28:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:28:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:28:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:28:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:28:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:28:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:28:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:28:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:28:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:28:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:28:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:28:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:28:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:28:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:28:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:29:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:29:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:29:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:29:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:29:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:29:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:29:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:29:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:29:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:29:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:29:09.805 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:29:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:29:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:29:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:29:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:29:39.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:29:39.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:29:39.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:29:39.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:29:39.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:29:39.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:29:39.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:29:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:29:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:29:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:29:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:29:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:29:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:29:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:29:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:29:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:29:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:29:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:29:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:29:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:29:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:30:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:30:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:30:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:30:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:30:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:30:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:30:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:30:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:30:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:30:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:30:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:30:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:30:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:30:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:30:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:30:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:30:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:30:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:30:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:30:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:30:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:30:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:30:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:30:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:30:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:30:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:30:49.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:30:49.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:30:49.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:30:49.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:30:49.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:30:49.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:30:49.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:30:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:30:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:30:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:31:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:31:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:31:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:31:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:31:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:31:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:31:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:31:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:31:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:31:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:31:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:31:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:31:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:31:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:31:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:31:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:31:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:31:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:31:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:31:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:31:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:31:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:31:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:31:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.059 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:31:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:31:49.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:31:49.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:31:49.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:31:49.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:31:49.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:31:49.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:31:49.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:31:49.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:31:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:31:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.096 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:31:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:32:09.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:32:09.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:32:09.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:32:09.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:32:09.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:32:09.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:32:09.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:32:09.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:32:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:32:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.133 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:32:09.833 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:32:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:32:39.686 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:33:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:33:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:33:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:33:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:33:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:33:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:33:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:33:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:33:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:33:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:33:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:33:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:33:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:33:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:33:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:33:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:33:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:33:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:33:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:33:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:33:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:33:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:33:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:33:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:33:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:34:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:34:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:34:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:34:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:34:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:34:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:34:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:34:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:34:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:34:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:34:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:34:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:34:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:34:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:34:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:34:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:34:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:34:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:34:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:34:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:34:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:34:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:34:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:34:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:34:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:34:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:34:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:34:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:34:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:34:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:34:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:34:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:34:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:34:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:34:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:34:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:35:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:35:09.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:35:09.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:35:09.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:35:09.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:35:09.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:35:09.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:35:09.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:35:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:35:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:35:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:35:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:35:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:35:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:35:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:35:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:35:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:35:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:35:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:35:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:35:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:35:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:35:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:35:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:35:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:35:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:35:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:35:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:35:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:35:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:35:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:35:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:35:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:35:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:35:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:35:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:36:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:36:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:36:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:36:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:36:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:36:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:36:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:36:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:36:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:36:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:36:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:36:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:36:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:36:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:36:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:36:39.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:36:39.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:36:39.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:36:39.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:36:39.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:36:39.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:36:39.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:36:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:36:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:36:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:36:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:36:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:36:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:36:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:36:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:36:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:36:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:36:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:36:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:36:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:36:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:37:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:37:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:37:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:37:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:37:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:37:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:37:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:37:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:37:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:37:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:37:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:37:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:37:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:37:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:37:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:37:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:37:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:37:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:37:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:37:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:37:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:37:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:37:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:37:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:37:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:37:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:37:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:37:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:37:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:37:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:37:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:37:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:37:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:37:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:37:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:37:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:38:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:38:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:38:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:38:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:38:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:38:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:38:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:38:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:38:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:38:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:38:09.808 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:38:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:38:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:38:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:38:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:38:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:38:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:38:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:38:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:38:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:38:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:38:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:38:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:38:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:38:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:38:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:38:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:38:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:38:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:38:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:38:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:38:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:38:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:38:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:38:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:38:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:39:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:39:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:39:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:39:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:39:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:39:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:39:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:39:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:39:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:39:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:39:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:39:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:39:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:39:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:39:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:39:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:39:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:39:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:39:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:39:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:39:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:39:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:39:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:39:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:39:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:39:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:39:49.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:39:49.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:39:49.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:39:49.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:39:49.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:39:49.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:39:49.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:39:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:39:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:39:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:40:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:40:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:40:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:40:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:40:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:40:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:40:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:40:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:40:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:40:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:40:09.799 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:40:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:40:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:40:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:40:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:40:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:40:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:40:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:40:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:40:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:40:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:40:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:40:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:40:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:40:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:40:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:40:49.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:40:49.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:40:49.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:40:49.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:40:49.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:40:49.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:40:49.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:40:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:40:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:40:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:41:09.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:41:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:41:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:41:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:41:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:41:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:41:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:41:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:41:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:41:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:41:09.805 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:41:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:41:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:41:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:41:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:41:39.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:41:39.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:41:39.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:41:39.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:41:39.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:41:39.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:41:39.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:41:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:41:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:41:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:41:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:41:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:41:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:41:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:41:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:41:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:41:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:41:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:41:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:41:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:41:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:42:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:42:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:42:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:42:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:42:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:42:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:42:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:42:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:42:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:42:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:42:09.791 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:42:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:42:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:42:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:42:39.679 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:42:39.712 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:42:39.712 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:42:39.712 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:42:39.712 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:42:39.712 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:42:39.712 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:42:39.712 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:42:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:42:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:42:39.714 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:42:49.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:42:49.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:42:49.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:42:49.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:42:49.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:42:49.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:42:49.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:42:49.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:42:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:42:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:42:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:43:09.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:43:09.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:43:09.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:43:09.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:43:09.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:43:09.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:43:09.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:43:09.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:43:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:43:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:43:09.789 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:43:39.670 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:43:39.671 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:43:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:43:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:43:39.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:43:39.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:43:39.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:43:39.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:43:39.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:43:39.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:43:39.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:43:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:43:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:43:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:43:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:43:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:43:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:43:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:43:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:43:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:43:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:43:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:43:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:43:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:43:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:44:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:44:09.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:44:09.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:44:09.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:44:09.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:44:09.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:44:09.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:44:09.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:44:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:44:09.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:44:09.788 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:44:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:44:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:44:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:44:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:44:39.709 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:44:39.709 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:44:39.709 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:44:39.709 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:44:39.709 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:44:39.709 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:44:39.709 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:44:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:44:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:44:39.712 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:44:49.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:44:49.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:44:49.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:44:49.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:44:49.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:44:49.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:44:49.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:44:49.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:44:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:44:49.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:44:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:45:09.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:45:09.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:45:09.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:45:09.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:45:09.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:45:09.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:45:09.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:45:09.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:45:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:45:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:45:09.787 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:45:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:45:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:45:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:45:39.677 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:45:39.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:45:39.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:45:39.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:45:39.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:45:39.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:45:39.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:45:39.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:45:39.715 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:45:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:45:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:45:49.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:45:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:45:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:45:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:45:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:45:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:45:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:45:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:45:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:45:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.096 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:45:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:46:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:46:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:46:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:46:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:46:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:46:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:46:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:46:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:46:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:46:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.133 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:46:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:46:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:46:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:46:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:46:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:46:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:46:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:46:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:46:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:46:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:46:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:46:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:46:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:46:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:46:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:46:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:46:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:46:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:46:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:46:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:46:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:46:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:46:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:46:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:46:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:46:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:47:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:47:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:47:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:47:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:47:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:47:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:47:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:47:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:47:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:47:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:47:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:47:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:47:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:47:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:47:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:47:39.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:47:39.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:47:39.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:47:39.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:47:39.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:47:39.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:47:39.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:47:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:47:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:47:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:47:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:47:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:47:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:47:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:47:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:47:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:47:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:47:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:47:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:47:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.089 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:47:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:48:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:48:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:48:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:48:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:48:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:48:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:48:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:48:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:48:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:48:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:48:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:48:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:48:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:48:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:48:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:48:39.713 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:48:39.713 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:48:39.713 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:48:39.713 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:48:39.713 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:48:39.713 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:48:39.713 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:48:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:48:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:48:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:48:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:48:49.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:48:49.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:48:49.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:48:49.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:48:49.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:48:49.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:48:49.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:48:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:48:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:48:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:49:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:49:09.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:49:09.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:49:09.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:49:09.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:49:09.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:49:09.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:49:09.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:49:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:49:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:49:09.788 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:49:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:49:39.670 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:49:39.703 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:49:39.703 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:49:39.703 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:49:39.703 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:49:39.703 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:49:39.703 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:49:39.703 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:49:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:49:39.705 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:49:39.706 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:49:49.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:49:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:49:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:49:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:49:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:49:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:49:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:49:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:49:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:49:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:49:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:50:09.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:50:09.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:50:09.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:50:09.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:50:09.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:50:09.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:50:09.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:50:09.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:50:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:50:09.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:50:09.788 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:50:39.669 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:50:39.670 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:50:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:50:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:50:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:50:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:50:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:50:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:50:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:50:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:50:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:50:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:50:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:50:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:50:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:50:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:50:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:50:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:50:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:50:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:50:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:50:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:50:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:50:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:50:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:51:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:51:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:51:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:51:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:51:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:51:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:51:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:51:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:51:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:51:09.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:51:09.791 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:51:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:51:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:51:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:51:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:51:39.715 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:51:39.715 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:51:39.715 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:51:39.715 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:51:39.715 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:51:39.715 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:51:39.715 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:51:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:51:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:51:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:51:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:51:49.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:51:49.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:51:49.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:51:49.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:51:49.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:51:49.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:51:49.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:51:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:51:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:51:49.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:52:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:52:09.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:52:09.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:52:09.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:52:09.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:52:09.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:52:09.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:52:09.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:52:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:52:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:52:09.794 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:52:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:52:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:52:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:52:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:52:39.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:52:39.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:52:39.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:52:39.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:52:39.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:52:39.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:52:39.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:52:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:52:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:52:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:52:49.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:52:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:52:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:52:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:52:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:52:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:52:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:52:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:52:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:52:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:52:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:53:09.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:53:09.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:53:09.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:53:09.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:53:09.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:53:09.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:53:09.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:53:09.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:53:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:53:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:53:09.801 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:53:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:53:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:53:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:53:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:53:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:53:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:53:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:53:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:53:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:53:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:53:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:53:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:53:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:53:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:53:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:53:49.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:53:49.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:53:49.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:53:49.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:53:49.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:53:49.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:53:49.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:53:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:53:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:53:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:54:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:54:09.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:54:09.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:54:09.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:54:09.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:54:09.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:54:09.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:54:09.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:54:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:54:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:54:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:54:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:54:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:54:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:54:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:54:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:54:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:54:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:54:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:54:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:54:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:54:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:54:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:54:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:54:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:54:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:54:49.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:54:49.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:54:49.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:54:49.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:54:49.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:54:49.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:54:49.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:54:49.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:54:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:54:49.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:55:09.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:55:09.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:55:09.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:55:09.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:55:09.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:55:09.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:55:09.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:55:09.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:55:09.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:55:09.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:55:09.790 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:55:39.671 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:55:39.672 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:55:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:55:39.680 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:55:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:55:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:55:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:55:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:55:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:55:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:55:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:55:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:55:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:55:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:55:49.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:55:49.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:55:49.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:55:49.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:55:49.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:55:49.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:55:49.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:55:49.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:55:49.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:55:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:55:49.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:56:09.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:56:09.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:56:09.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:56:09.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:56:09.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:56:09.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:56:09.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:56:09.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:56:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:56:09.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:56:09.793 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:56:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:56:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:56:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:56:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:56:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:56:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:56:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:56:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:56:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:56:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:56:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:56:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:56:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:56:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:56:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:56:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:56:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:56:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:56:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:56:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:56:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:56:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:56:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:56:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:56:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:57:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:57:09.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:57:09.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:57:09.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:57:09.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:57:09.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:57:09.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:57:09.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:57:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:57:09.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:57:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:57:39.679 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:57:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:57:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:57:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:57:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:57:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:57:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:57:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:57:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:57:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:57:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:57:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:57:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:57:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:57:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:57:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:57:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:57:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:57:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:57:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:57:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:57:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:57:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:57:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:57:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:58:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:58:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:58:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:58:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:58:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:58:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:58:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:58:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:58:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:58:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:58:09.803 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:58:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:58:39.675 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:58:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:58:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:58:39.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:58:39.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:58:39.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:58:39.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:58:39.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:58:39.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:58:39.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:58:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:58:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:58:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:58:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:58:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:58:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:58:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:58:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:58:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:58:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:58:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:58:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:58:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:58:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:59:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:59:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:59:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:59:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:59:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:59:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:59:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:59:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:59:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:59:09.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 00:59:09.795 139328 ERROR oslo_service.periodic_task 
>2018-06-25 00:59:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:59:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 00:59:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 00:59:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:59:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:59:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:59:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:59:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:59:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:59:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:59:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:59:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:59:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:59:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 00:59:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 00:59:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 00:59:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 00:59:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 00:59:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 00:59:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 00:59:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 00:59:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 00:59:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 00:59:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 00:59:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:00:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:00:09.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:00:09.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:00:09.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:00:09.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:00:09.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:00:09.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:00:09.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:00:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:00:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:00:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:00:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:00:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:00:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:00:39.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:00:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:00:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:00:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:00:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:00:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:00:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:00:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:00:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:00:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:00:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:00:49.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:00:49.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:00:49.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:00:49.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:00:49.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:00:49.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:00:49.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:00:49.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:00:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:00:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:00:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:01:09.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:01:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:01:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:01:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:01:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:01:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:01:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:01:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:01:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:01:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:01:09.802 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:01:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:01:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:01:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:01:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:01:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:01:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:01:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:01:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:01:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:01:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:01:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:01:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:01:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:01:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:01:49.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:01:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:01:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:01:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:01:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:01:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:01:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:01:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:01:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:01:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:01:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:02:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:02:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:02:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:02:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:02:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:02:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:02:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:02:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:02:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:02:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:02:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:02:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:02:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:02:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:02:39.691 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:02:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:02:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:02:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:02:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:02:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:02:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:02:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:02:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:02:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:02:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:02:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:02:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:02:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:02:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:02:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:02:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:02:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:02:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:02:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:02:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:02:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:03:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:03:09.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:03:09.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:03:09.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:03:09.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:03:09.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:03:09.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:03:09.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:03:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:03:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:03:09.801 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:03:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:03:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:03:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:03:39.693 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:03:39.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:03:39.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:03:39.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:03:39.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:03:39.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:03:39.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:03:39.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:03:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:03:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:03:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:03:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:03:49.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:03:49.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:03:49.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:03:49.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:03:49.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:03:49.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:03:49.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:03:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:03:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.089 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:03:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:04:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:04:09.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:04:09.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:04:09.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:04:09.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:04:09.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:04:09.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:04:09.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:04:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:04:09.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:04:09.818 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:04:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:04:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:04:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:04:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:04:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:04:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:04:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:04:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:04:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:04:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:04:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:04:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:04:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:04:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:04:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:04:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:04:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:04:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:04:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:04:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:04:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:04:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:04:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:04:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:04:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:05:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:05:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:05:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:05:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:05:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:05:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:05:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:05:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:05:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:05:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:05:09.798 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:05:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:05:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:05:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:05:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:05:39.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:05:39.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:05:39.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:05:39.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:05:39.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:05:39.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:05:39.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:05:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:05:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:05:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:05:49.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:05:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:05:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:05:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:05:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:05:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:05:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:05:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:05:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:05:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:05:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:06:09.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:06:09.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:06:09.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:06:09.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:06:09.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:06:09.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:06:09.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:06:09.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:06:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:06:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:06:09.801 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:06:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:06:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:06:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:06:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:06:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:06:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:06:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:06:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:06:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:06:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:06:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:06:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:06:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:06:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:06:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:06:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:06:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:06:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:06:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:06:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:06:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:06:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:06:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:06:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:06:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:07:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:07:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:07:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:07:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:07:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:07:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:07:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:07:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:07:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:07:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:07:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:07:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:07:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:07:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:07:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:07:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:07:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:07:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:07:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:07:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:07:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:07:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:07:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:07:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:07:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:07:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:07:49.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:07:49.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:07:49.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:07:49.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:07:49.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:07:49.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:07:49.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:07:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:07:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:07:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:08:09.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:08:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:08:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:08:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:08:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:08:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:08:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:08:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:08:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:08:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:08:09.808 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:08:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:08:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:08:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:08:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:08:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:08:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:08:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:08:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:08:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:08:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:08:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:08:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:08:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:08:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:08:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:08:49.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:08:49.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:08:49.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:08:49.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:08:49.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:08:49.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:08:49.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:08:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:08:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:08:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:09:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:09:09.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:09:09.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:09:09.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:09:09.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:09:09.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:09:09.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:09:09.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:09:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:09:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:09:09.814 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:09:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:09:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:09:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:09:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:09:39.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:09:39.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:09:39.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:09:39.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:09:39.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:09:39.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:09:39.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:09:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:09:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:09:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:09:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:09:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:09:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:09:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:09:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:09:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:09:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:09:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:09:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:09:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:09:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:10:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:10:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:10:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:10:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:10:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:10:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:10:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:10:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:10:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:10:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:10:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:10:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:10:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:10:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:10:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:10:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:10:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:10:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:10:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:10:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:10:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:10:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:10:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:10:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:10:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:10:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:10:49.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:10:49.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:10:49.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:10:49.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:10:49.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:10:49.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:10:49.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:10:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:10:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:10:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:11:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:11:09.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:11:09.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:11:09.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:11:09.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:11:09.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:11:09.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:11:09.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:11:09.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:11:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.128 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:11:09.826 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:11:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:11:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:11:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:11:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:11:39.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:11:39.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:11:39.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:11:39.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:11:39.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:11:39.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:11:39.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:11:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:11:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:11:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:11:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:11:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:11:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:11:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:11:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:11:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:11:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:11:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:11:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:11:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:11:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:12:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:12:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:12:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:12:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:12:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:12:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:12:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:12:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:12:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:12:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:12:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:12:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:12:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:12:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:12:39.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:12:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:12:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:12:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:12:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:12:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:12:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:12:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:12:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:12:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:12:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:12:49.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:12:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:12:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:12:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:12:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:12:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:12:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:12:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:12:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:12:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:12:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:13:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:13:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:13:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:13:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:13:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:13:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:13:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:13:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:13:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:13:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:13:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:13:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:13:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:13:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:13:39.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:13:39.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:13:39.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:13:39.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:13:39.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:13:39.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:13:39.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:13:39.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:13:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:13:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:13:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:13:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:13:49.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:13:49.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:13:49.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:13:49.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:13:49.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:13:49.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:13:49.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:13:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:13:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:13:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:14:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:14:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:14:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:14:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:14:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:14:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:14:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:14:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:14:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:14:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:14:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:14:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:14:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:14:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:14:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:14:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:14:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:14:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:14:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:14:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:14:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:14:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:14:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:14:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:14:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:14:49.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:14:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:14:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:14:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:14:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:14:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:14:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:14:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:14:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:14:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:14:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:15:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:15:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:15:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:15:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:15:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:15:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:15:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:15:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:15:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:15:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:15:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:15:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:15:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:15:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:15:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:15:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:15:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:15:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:15:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:15:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:15:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:15:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:15:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:15:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:15:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:15:49.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:15:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:15:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:15:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:15:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:15:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:15:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:15:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:15:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:15:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:15:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:16:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:16:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:16:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:16:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:16:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:16:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:16:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:16:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:16:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:16:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:16:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:16:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:16:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:16:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:16:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:16:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:16:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:16:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:16:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:16:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:16:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:16:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:16:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:16:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:16:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:16:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:16:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:16:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:16:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:16:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:16:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:16:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:16:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:16:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:16:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:16:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:17:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:17:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:17:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:17:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:17:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:17:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:17:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:17:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:17:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:17:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:17:09.825 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:17:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:17:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:17:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:17:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:17:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:17:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:17:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:17:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:17:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:17:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:17:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:17:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:17:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:17:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:17:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:17:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:17:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:17:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:17:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:17:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:17:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:17:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:17:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:17:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:17:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:18:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:18:09.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:18:09.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:18:09.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:18:09.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:18:09.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:18:09.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:18:09.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:18:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:18:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:18:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:18:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:18:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:18:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:18:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:18:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:18:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:18:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:18:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:18:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:18:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:18:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:18:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:18:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:18:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:18:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:18:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:18:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:18:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:18:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:18:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:18:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:18:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:18:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:18:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:18:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:19:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:19:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:19:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:19:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:19:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:19:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:19:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:19:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:19:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:19:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:19:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:19:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:19:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:19:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:19:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:19:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:19:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:19:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:19:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:19:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:19:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:19:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:19:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:19:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:19:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:19:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:19:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:19:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:19:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:19:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:19:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:19:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:19:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:19:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:19:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:19:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:20:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:20:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:20:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:20:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:20:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:20:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:20:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:20:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:20:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:20:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:20:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:20:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:20:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:20:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:20:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:20:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:20:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:20:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:20:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:20:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:20:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:20:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:20:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:20:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:20:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:20:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:20:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:20:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:20:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:20:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:20:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:20:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:20:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:20:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:20:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:20:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:21:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:21:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:21:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:21:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:21:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:21:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:21:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:21:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:21:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:21:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:21:09.814 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:21:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:21:39.697 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:21:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:21:39.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:21:39.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:21:39.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:21:39.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:21:39.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:21:39.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:21:39.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:21:39.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:21:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:21:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.047 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:21:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:21:49.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:21:49.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:21:49.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:21:49.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:21:49.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:21:49.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:21:49.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:21:49.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:21:49.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:21:49.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.089 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:21:49.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:22:09.821 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:22:09.864 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:22:09.864 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:22:09.864 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:22:09.864 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:22:09.864 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:22:09.864 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:22:09.864 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:22:09.866 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:22:09.867 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.136 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:22:09.867 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:22:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:22:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:22:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:22:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:22:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:22:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:22:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:22:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:22:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:22:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:22:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:22:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:22:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:22:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:22:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:22:49.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:22:49.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:22:49.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:22:49.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:22:49.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:22:49.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:22:49.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:22:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:22:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:22:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:23:09.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:23:09.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:23:09.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:23:09.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:23:09.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:23:09.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:23:09.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:23:09.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:23:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:23:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:23:09.827 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:23:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:23:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:23:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:23:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:23:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:23:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:23:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:23:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:23:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:23:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:23:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:23:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:23:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:23:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:23:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:23:49.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:23:49.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:23:49.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:23:49.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:23:49.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:23:49.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:23:49.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:23:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:23:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:23:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:24:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:24:09.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:24:09.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:24:09.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:24:09.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:24:09.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:24:09.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:24:09.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:24:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:24:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:24:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:24:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:24:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:24:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:24:39.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:24:39.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:24:39.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:24:39.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:24:39.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:24:39.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:24:39.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:24:39.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:24:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:24:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:24:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:24:49.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:24:49.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:24:49.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:24:49.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:24:49.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:24:49.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:24:49.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:24:49.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:24:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:24:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:24:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:25:09.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:25:09.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:25:09.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:25:09.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:25:09.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:25:09.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:25:09.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:25:09.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:25:09.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:25:09.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:25:09.833 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:25:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:25:39.705 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:25:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:25:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:25:39.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:25:39.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:25:39.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:25:39.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:25:39.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:25:39.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:25:39.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:25:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:25:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.051 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:25:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:25:49.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:25:49.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:25:49.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:25:49.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:25:49.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:25:49.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:25:49.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:25:49.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:25:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:25:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.087 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:25:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:26:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:26:09.838 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:26:09.838 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:26:09.838 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:26:09.838 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:26:09.838 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:26:09.838 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:26:09.838 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:26:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:26:09.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:26:09.843 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:26:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:26:39.707 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:26:39.714 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:26:39.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:26:39.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:26:39.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:26:39.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:26:39.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:26:39.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:26:39.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:26:39.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:26:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:26:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:26:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:26:49.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:26:49.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:26:49.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:26:49.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:26:49.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:26:49.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:26:49.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:26:49.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:26:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:26:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:26:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:27:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:27:09.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:27:09.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:27:09.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:27:09.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:27:09.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:27:09.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:27:09.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:27:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:27:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:27:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:27:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:27:39.704 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:27:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:27:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:27:39.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:27:39.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:27:39.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:27:39.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:27:39.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:27:39.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:27:39.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:27:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:27:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:27:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:27:49.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:27:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:27:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:27:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:27:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:27:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:27:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:27:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:27:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:27:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:27:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:28:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:28:09.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:28:09.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:28:09.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:28:09.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:28:09.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:28:09.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:28:09.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:28:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:28:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:28:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:28:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:28:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:28:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:28:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:28:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:28:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:28:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:28:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:28:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:28:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:28:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:28:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:28:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:28:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:28:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:28:49.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:28:49.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:28:49.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:28:49.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:28:49.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:28:49.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:28:49.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:28:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:28:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:28:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:29:09.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:29:09.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:29:09.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:29:09.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:29:09.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:29:09.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:29:09.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:29:09.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:29:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:29:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:29:09.818 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:29:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:29:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:29:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:29:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:29:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:29:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:29:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:29:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:29:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:29:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:29:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:29:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:29:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:29:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:29:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:29:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:29:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:29:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:29:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:29:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:29:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:29:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:29:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:29:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:29:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:30:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:30:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:30:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:30:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:30:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:30:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:30:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:30:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:30:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:30:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:30:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:30:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:30:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:30:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:30:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:30:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:30:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:30:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:30:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:30:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:30:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:30:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:30:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:30:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:30:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:30:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:30:49.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:30:49.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:30:49.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:30:49.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:30:49.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:30:49.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:30:49.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:30:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:30:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:30:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:31:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:31:09.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:31:09.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:31:09.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:31:09.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:31:09.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:31:09.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:31:09.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:31:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:31:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:31:09.834 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:31:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:31:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:31:39.714 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:31:39.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:31:39.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:31:39.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:31:39.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:31:39.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:31:39.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:31:39.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:31:39.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:31:39.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:31:39.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.055 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:31:39.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:31:49.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:31:49.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:31:49.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:31:49.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:31:49.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:31:49.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:31:49.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:31:49.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:31:49.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:31:49.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.094 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:31:49.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:32:09.810 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:32:09.845 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:32:09.845 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:32:09.845 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:32:09.845 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:32:09.845 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:32:09.845 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:32:09.845 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:32:09.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:32:09.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.132 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:32:09.848 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:32:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:32:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:32:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:32:39.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:32:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:32:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:32:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:32:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:32:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:32:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:32:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:32:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:32:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:32:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:32:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:32:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:32:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:32:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:32:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:32:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:32:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:32:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:32:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:32:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:32:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:33:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:33:09.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:33:09.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:33:09.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:33:09.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:33:09.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:33:09.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:33:09.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:33:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:33:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:33:09.830 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:33:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:33:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:33:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:33:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:33:39.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:33:39.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:33:39.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:33:39.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:33:39.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:33:39.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:33:39.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:33:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:33:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:33:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:33:49.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:33:49.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:33:49.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:33:49.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:33:49.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:33:49.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:33:49.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:33:49.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:33:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:33:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:33:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:34:09.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:34:09.833 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:34:09.833 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:34:09.833 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:34:09.833 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:34:09.833 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:34:09.833 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:34:09.833 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:34:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:34:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:34:09.836 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:34:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:34:39.709 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:34:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:34:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:34:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:34:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:34:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:34:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:34:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:34:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:34:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:34:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:34:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:34:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:34:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:34:49.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:34:49.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:34:49.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:34:49.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:34:49.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:34:49.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:34:49.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:34:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:34:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:34:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:35:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:35:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:35:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:35:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:35:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:35:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:35:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:35:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:35:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:35:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:35:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:35:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:35:39.711 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:35:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:35:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:35:39.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:35:39.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:35:39.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:35:39.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:35:39.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:35:39.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:35:39.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:35:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:35:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:35:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:35:49.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:35:49.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:35:49.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:35:49.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:35:49.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:35:49.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:35:49.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:35:49.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:35:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:35:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:35:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:36:09.799 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:36:09.833 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:36:09.833 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:36:09.833 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:36:09.833 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:36:09.833 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:36:09.833 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:36:09.833 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:36:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:36:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:36:09.836 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:36:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:36:39.709 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:36:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:36:39.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:36:39.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:36:39.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:36:39.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:36:39.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:36:39.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:36:39.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:36:39.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:36:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:36:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:36:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:36:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:36:49.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:36:49.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:36:49.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:36:49.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:36:49.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:36:49.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:36:49.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:36:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:36:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:36:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:37:09.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:37:09.835 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:37:09.835 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:37:09.835 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:37:09.835 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:37:09.835 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:37:09.835 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:37:09.835 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:37:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:37:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:37:09.838 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:37:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:37:39.711 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:37:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:37:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:37:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:37:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:37:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:37:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:37:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:37:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:37:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:37:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:37:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:37:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:37:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:37:49.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:37:49.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:37:49.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:37:49.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:37:49.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:37:49.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:37:49.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:37:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:37:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:37:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:38:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:38:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:38:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:38:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:38:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:38:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:38:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:38:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:38:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:38:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:38:09.827 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:38:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:38:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:38:39.717 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:38:39.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:38:39.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:38:39.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:38:39.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:38:39.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:38:39.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:38:39.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:38:39.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:38:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:38:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:38:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:38:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:38:49.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:38:49.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:38:49.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:38:49.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:38:49.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:38:49.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:38:49.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:38:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:38:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:38:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:39:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:39:09.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:39:09.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:39:09.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:39:09.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:39:09.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:39:09.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:39:09.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:39:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:39:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:39:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:39:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:39:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:39:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:39:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:39:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:39:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:39:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:39:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:39:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:39:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:39:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:39:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:39:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:39:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:39:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:39:49.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:39:49.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:39:49.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:39:49.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:39:49.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:39:49.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:39:49.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:39:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:39:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:39:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:40:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:40:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:40:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:40:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:40:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:40:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:40:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:40:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:40:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:40:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:40:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:40:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:40:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:40:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:40:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:40:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:40:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:40:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:40:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:40:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:40:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:40:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:40:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:40:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:40:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:40:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:40:49.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:40:49.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:40:49.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:40:49.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:40:49.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:40:49.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:40:49.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:40:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:40:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:40:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:41:09.793 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:41:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:41:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:41:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:41:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:41:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:41:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:41:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:41:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:41:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:41:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:41:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:41:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:41:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:41:39.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:41:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:41:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:41:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:41:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:41:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:41:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:41:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:41:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:41:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:41:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:41:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:41:49.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:41:49.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:41:49.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:41:49.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:41:49.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:41:49.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:41:49.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:41:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:41:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:41:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:42:09.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:42:09.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:42:09.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:42:09.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:42:09.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:42:09.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:42:09.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:42:09.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:42:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:42:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:42:09.835 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:42:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:42:39.707 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:42:39.717 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:42:39.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:42:39.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:42:39.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:42:39.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:42:39.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:42:39.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:42:39.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:42:39.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:42:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:42:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:42:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:42:49.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:42:49.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:42:49.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:42:49.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:42:49.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:42:49.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:42:49.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:42:49.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:42:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:42:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:42:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:43:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:43:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:43:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:43:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:43:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:43:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:43:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:43:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:43:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:43:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:43:09.827 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:43:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:43:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:43:39.717 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:43:39.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:43:39.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:43:39.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:43:39.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:43:39.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:43:39.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:43:39.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:43:39.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:43:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:43:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:43:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:43:49.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:43:49.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:43:49.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:43:49.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:43:49.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:43:49.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:43:49.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:43:49.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:43:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:43:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:43:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:44:09.799 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:44:09.832 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:44:09.832 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:44:09.832 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:44:09.832 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:44:09.832 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:44:09.832 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:44:09.832 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:44:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:44:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:44:09.835 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:44:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:44:39.707 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:45:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:45:39.705 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:45:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:45:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:45:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:45:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:45:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:45:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:45:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:45:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:45:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:45:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:45:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:45:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:45:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:45:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:45:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:45:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:45:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:45:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:45:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:45:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:45:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:45:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:45:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:46:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:46:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:46:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:46:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:46:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:46:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:46:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:46:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:46:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:46:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:46:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:46:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:46:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:46:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:46:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:46:39.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:46:39.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:46:39.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:46:39.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:46:39.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:46:39.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:46:39.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:46:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:46:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:46:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:46:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:46:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:46:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:46:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:46:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:46:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:46:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:46:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:46:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:46:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:46:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:47:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:47:09.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:47:09.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:47:09.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:47:09.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:47:09.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:47:09.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:47:09.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:47:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:47:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:47:09.825 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:47:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:47:39.707 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:47:39.715 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:47:39.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:47:39.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:47:39.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:47:39.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:47:39.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:47:39.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:47:39.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:47:39.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:47:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:47:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:47:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:47:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:47:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:47:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:47:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:47:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:47:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:47:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:47:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:47:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:47:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:47:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:48:09.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:48:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:48:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:48:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:48:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:48:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:48:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:48:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:48:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:48:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:48:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:48:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:48:39.704 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:48:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:48:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:48:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:48:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:48:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:48:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:48:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:48:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:48:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:48:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:48:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:48:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:48:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:48:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:48:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:48:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:48:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:48:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:48:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:48:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:48:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:48:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:48:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:49:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:49:09.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:49:09.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:49:09.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:49:09.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:49:09.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:49:09.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:49:09.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:49:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:49:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:49:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:49:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:49:39.703 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:49:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:49:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:49:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:49:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:49:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:49:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:49:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:49:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:49:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:49:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:49:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:49:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:49:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:49:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:49:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:49:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:49:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:49:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:49:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:49:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:49:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:49:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:49:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:50:09.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:50:09.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:50:09.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:50:09.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:50:09.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:50:09.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:50:09.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:50:09.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:50:09.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:50:09.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:50:09.826 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:50:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:50:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:50:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:50:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:50:39.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:50:39.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:50:39.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:50:39.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:50:39.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:50:39.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:50:39.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:50:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:50:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:50:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:50:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:50:49.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:50:49.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:50:49.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:50:49.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:50:49.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:50:49.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:50:49.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:50:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:50:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:50:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:51:09.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:51:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:51:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:51:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:51:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:51:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:51:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:51:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:51:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:51:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:51:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:51:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:51:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:51:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:51:39.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:51:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:51:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:51:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:51:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:51:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:51:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:51:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:51:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:51:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:51:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:51:49.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:51:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:51:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:51:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:51:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:51:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:51:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:51:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:51:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:51:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:51:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:52:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:52:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:52:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:52:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:52:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:52:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:52:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:52:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:52:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:52:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:52:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:52:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:52:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:52:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:52:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:52:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:52:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:52:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:52:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:52:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:52:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:52:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:52:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:52:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:52:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:52:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:52:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:52:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:52:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:52:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:52:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:52:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:52:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:52:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:52:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:52:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:53:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:53:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:53:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:53:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:53:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:53:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:53:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:53:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:53:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:53:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:53:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:53:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:53:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:53:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:53:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:53:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:53:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:53:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:53:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:53:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:53:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:53:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:53:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:53:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:53:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:53:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:53:49.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:53:49.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:53:49.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:53:49.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:53:49.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:53:49.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:53:49.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:53:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:53:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:53:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:54:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:54:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:54:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:54:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:54:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:54:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:54:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:54:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:54:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:54:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:54:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:54:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:54:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:54:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:54:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:54:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:54:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:54:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:54:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:54:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:54:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:54:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:54:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:54:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.047 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:54:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:54:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:54:49.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:54:49.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:54:49.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:54:49.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:54:49.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:54:49.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:54:49.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:54:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:54:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:54:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:55:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:55:09.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:55:09.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:55:09.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:55:09.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:55:09.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:55:09.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:55:09.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:55:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:55:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:55:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:55:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:55:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:55:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:55:39.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:55:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:55:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:55:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:55:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:55:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:55:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:55:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:55:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:55:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:55:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:55:49.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:55:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:55:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:55:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:55:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:55:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:55:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:55:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:55:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:55:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:55:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:56:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:56:09.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:56:09.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:56:09.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:56:09.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:56:09.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:56:09.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:56:09.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:56:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:56:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:56:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:56:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:56:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:56:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:56:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:56:39.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:56:39.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:56:39.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:56:39.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:56:39.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:56:39.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:56:39.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:56:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:56:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:56:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:56:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:56:49.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:56:49.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:56:49.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:56:49.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:56:49.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:56:49.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:56:49.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:56:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:56:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:56:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:57:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:57:09.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:57:09.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:57:09.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:57:09.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:57:09.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:57:09.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:57:09.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:57:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:57:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:57:09.814 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:57:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:57:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:57:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:57:39.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:57:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:57:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:57:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:57:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:57:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:57:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:57:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:57:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:57:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:57:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:57:49.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:57:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:57:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:57:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:57:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:57:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:57:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:57:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:57:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:57:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.09 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:57:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:58:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:58:09.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:58:09.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:58:09.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:58:09.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:58:09.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:58:09.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:58:09.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:58:09.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:58:09.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:58:09.833 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:58:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:58:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:58:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:58:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:58:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:58:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:58:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:58:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:58:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:58:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:58:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:58:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:58:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:58:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:58:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:58:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:58:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:58:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:58:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:58:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:58:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:58:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:58:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:59:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:59:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:59:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:59:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:59:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:59:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:59:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:59:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:59:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:59:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 01:59:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-25 01:59:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:59:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 01:59:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 01:59:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:59:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:59:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:59:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:59:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:59:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:59:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:59:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:59:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:59:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:59:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 01:59:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 01:59:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 01:59:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 01:59:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 01:59:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 01:59:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 01:59:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 01:59:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 01:59:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 01:59:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 01:59:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:00:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:00:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:00:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:00:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:00:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:00:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:00:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:00:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:00:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:00:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:00:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:00:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:00:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:00:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:00:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:00:39.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:00:39.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:00:39.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:00:39.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:00:39.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:00:39.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:00:39.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:00:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:00:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:00:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:00:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:00:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:00:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:00:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:00:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:00:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:00:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:00:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:00:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:00:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:00:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:01:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:01:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:01:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:01:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:01:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:01:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:01:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:01:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:01:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:01:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:01:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:01:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:01:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:01:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:01:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:01:39.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:01:39.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:01:39.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:01:39.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:01:39.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:01:39.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:01:39.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:01:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:01:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:01:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:01:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:01:49.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:01:49.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:01:49.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:01:49.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:01:49.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:01:49.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:01:49.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:01:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:01:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:01:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:02:09.793 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:02:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:02:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:02:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:02:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:02:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:02:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:02:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:02:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:02:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:02:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:02:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:02:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:02:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:02:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:02:39.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:02:39.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:02:39.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:02:39.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:02:39.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:02:39.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:02:39.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:02:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:02:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:02:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:02:49.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:02:49.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:02:49.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:02:49.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:02:49.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:02:49.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:02:49.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:02:49.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:02:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:02:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:02:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:03:09.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:03:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:03:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:03:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:03:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:03:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:03:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:03:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:03:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:03:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:03:09.830 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:03:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:03:39.703 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:03:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:03:39.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:03:39.744 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:03:39.744 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:03:39.744 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:03:39.744 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:03:39.744 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:03:39.744 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:03:39.744 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:03:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:03:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:03:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:03:49.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:03:49.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:03:49.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:03:49.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:03:49.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:03:49.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:03:49.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:03:49.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:03:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:03:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:03:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:04:09.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:04:09.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:04:09.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:04:09.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:04:09.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:04:09.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:04:09.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:04:09.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:04:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:04:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:04:09.834 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:04:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:04:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:04:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:04:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:04:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:04:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:04:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:04:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:04:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:04:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:04:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:04:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:04:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:04:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:04:49.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:04:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:04:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:04:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:04:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:04:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:04:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:04:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:04:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:04:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:04:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:05:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:05:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:05:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:05:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:05:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:05:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:05:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:05:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:05:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:05:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:05:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:05:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:05:39.704 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:05:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:05:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:05:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:05:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:05:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:05:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:05:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:05:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:05:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:05:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:05:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.031 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:05:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:05:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:05:49.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:05:49.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:05:49.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:05:49.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:05:49.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:05:49.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:05:49.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:05:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:05:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:05:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:06:09.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:06:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:06:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:06:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:06:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:06:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:06:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:06:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:06:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:06:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.104 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:06:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:06:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:06:39.709 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:06:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:06:39.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:06:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:06:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:06:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:06:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:06:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:06:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:06:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:06:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:06:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:06:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:06:49.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:06:49.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:06:49.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:06:49.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:06:49.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:06:49.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:06:49.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:06:49.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:06:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:06:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:06:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:07:09.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:07:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:07:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:07:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:07:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:07:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:07:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:07:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:07:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:07:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:07:09.839 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:07:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:07:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:07:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:07:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:07:39.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:07:39.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:07:39.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:07:39.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:07:39.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:07:39.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:07:39.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:07:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:07:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:07:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:07:49.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:07:49.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:07:49.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:07:49.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:07:49.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:07:49.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:07:49.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:07:49.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:07:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:07:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:07:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:08:09.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:08:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:08:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:08:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:08:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:08:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:08:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:08:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:08:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:08:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:08:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:08:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:08:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:08:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:08:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:08:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:08:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:08:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:08:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:08:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:08:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:08:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:08:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:08:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:08:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:08:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:08:49.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:08:49.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:08:49.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:08:49.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:08:49.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:08:49.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:08:49.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:08:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:08:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:08:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:09:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:09:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:09:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:09:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:09:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:09:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:09:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:09:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:09:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:09:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:09:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:09:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:09:39.711 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:09:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:09:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:09:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:09:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:09:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:09:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:09:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:09:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:09:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:09:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:09:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:09:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:09:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:09:49.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:09:49.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:09:49.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:09:49.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:09:49.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:09:49.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:09:49.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:09:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:09:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:09:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:10:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:10:09.832 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:10:09.832 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:10:09.832 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:10:09.832 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:10:09.832 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:10:09.832 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:10:09.832 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:10:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:10:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:10:09.836 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:10:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:10:39.708 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:10:39.715 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:10:39.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:10:39.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:10:39.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:10:39.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:10:39.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:10:39.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:10:39.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:10:39.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:10:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:10:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:10:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:10:49.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:10:49.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:10:49.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:10:49.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:10:49.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:10:49.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:10:49.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:10:49.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:10:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:10:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:10:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:11:09.793 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:11:09.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:11:09.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:11:09.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:11:09.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:11:09.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:11:09.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:11:09.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:11:09.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:11:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:11:09.833 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:11:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:11:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:11:39.714 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:11:39.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:11:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:11:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:11:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:11:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:11:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:11:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:11:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:11:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:11:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:11:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:11:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:11:49.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:11:49.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:11:49.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:11:49.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:11:49.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:11:49.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:11:49.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:11:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:11:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.087 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:11:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:12:09.802 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:12:09.835 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:12:09.835 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:12:09.835 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:12:09.835 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:12:09.835 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:12:09.835 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:12:09.835 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:12:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:12:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:12:09.838 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:12:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:12:39.711 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:12:39.717 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:12:39.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:12:39.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:12:39.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:12:39.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:12:39.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:12:39.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:12:39.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:12:39.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:12:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:12:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:12:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:12:49.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:12:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:12:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:12:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:12:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:12:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:12:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:12:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:12:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:12:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:12:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:13:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:13:09.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:13:09.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:13:09.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:13:09.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:13:09.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:13:09.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:13:09.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:13:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:13:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:13:09.833 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:13:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:13:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:13:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:13:39.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:13:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:13:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:13:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:13:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:13:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:13:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:13:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:13:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:13:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:13:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:13:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:13:49.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:13:49.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:13:49.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:13:49.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:13:49.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:13:49.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:13:49.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:13:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:13:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:13:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:14:09.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:14:09.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:14:09.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:14:09.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:14:09.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:14:09.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:14:09.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:14:09.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:14:09.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:14:09.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:14:09.825 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:14:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:14:39.707 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:14:39.714 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:14:39.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:14:39.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:14:39.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:14:39.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:14:39.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:14:39.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:14:39.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:14:39.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:14:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:14:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:14:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:14:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:14:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:14:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:14:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:14:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:14:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:14:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:14:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:14:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:14:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:14:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:15:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:15:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:15:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:15:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:15:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:15:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:15:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:15:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:15:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:15:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:15:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:15:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:15:39.707 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:15:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:15:39.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:15:39.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:15:39.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:15:39.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:15:39.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:15:39.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:15:39.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:15:39.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:15:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:15:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:15:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:15:49.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:15:49.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:15:49.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:15:49.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:15:49.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:15:49.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:15:49.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:15:49.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:15:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:15:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:15:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:16:09.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:16:09.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:16:09.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:16:09.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:16:09.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:16:09.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:16:09.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:16:09.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:16:09.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:16:09.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:16:09.825 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:16:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:16:39.709 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:16:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:16:39.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:16:39.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:16:39.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:16:39.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:16:39.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:16:39.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:16:39.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:16:39.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:16:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:16:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:16:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:16:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:16:49.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:16:49.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:16:49.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:16:49.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:16:49.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:16:49.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:16:49.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:16:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:16:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:16:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:17:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:17:09.834 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:17:09.834 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:17:09.834 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:17:09.834 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:17:09.834 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:17:09.834 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:17:09.834 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:17:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:17:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:17:09.837 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:17:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:17:39.709 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:17:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:17:39.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:17:39.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:17:39.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:17:39.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:17:39.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:17:39.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:17:39.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:17:39.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:17:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:17:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:17:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:17:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:17:49.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:17:49.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:17:49.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:17:49.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:17:49.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:17:49.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:17:49.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:17:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:17:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:17:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:18:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:18:09.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:18:09.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:18:09.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:18:09.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:18:09.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:18:09.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:18:09.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:18:09.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:18:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:18:09.826 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:18:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:18:39.709 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:18:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:18:39.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:18:39.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:18:39.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:18:39.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:18:39.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:18:39.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:18:39.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:18:39.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:18:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:18:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:18:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:18:49.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:18:49.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:18:49.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:18:49.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:18:49.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:18:49.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:18:49.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:18:49.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:18:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:18:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:18:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:19:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:19:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:19:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:19:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:19:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:19:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:19:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:19:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:19:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:19:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:19:09.830 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:19:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:19:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:19:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:19:39.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:19:39.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:19:39.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:19:39.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:19:39.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:19:39.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:19:39.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:19:39.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:19:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:19:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:19:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:19:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:19:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:19:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:19:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:19:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:19:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:19:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:19:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:19:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:19:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:19:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:20:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:20:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:20:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:20:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:20:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:20:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:20:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:20:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:20:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:20:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:20:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:20:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:20:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:20:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:20:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:20:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:20:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:20:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:20:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:20:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:20:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:20:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:20:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:20:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:20:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:20:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:20:49.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:20:49.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:20:49.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:20:49.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:20:49.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:20:49.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:20:49.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:20:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:20:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:20:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:21:09.803 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:21:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:21:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:21:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:21:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:21:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:21:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:21:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:21:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:21:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:21:09.840 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:21:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:21:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:21:39.721 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:21:39.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:21:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:21:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:21:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:21:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:21:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:21:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:21:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:21:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:21:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:21:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:21:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:21:49.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:21:49.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:21:49.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:21:49.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:21:49.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:21:49.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:21:49.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:21:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:21:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:21:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:22:09.793 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:22:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:22:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:22:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:22:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:22:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:22:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:22:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:22:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:22:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:22:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:22:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:22:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:22:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:22:39.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:22:39.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:22:39.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:22:39.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:22:39.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:22:39.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:22:39.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:22:39.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:22:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:22:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:22:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:22:49.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:22:49.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:22:49.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:22:49.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:22:49.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:22:49.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:22:49.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:22:49.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:22:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:22:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:22:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:23:09.800 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:23:09.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:23:09.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:23:09.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:23:09.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:23:09.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:23:09.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:23:09.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:23:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:23:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:23:09.834 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:23:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:23:39.716 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:23:39.723 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:23:39.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:23:39.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:23:39.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:23:39.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:23:39.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:23:39.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:23:39.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:23:39.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:23:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:23:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:23:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:23:49.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:23:49.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:23:49.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:23:49.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:23:49.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:23:49.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:23:49.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:23:49.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:23:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:23:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:23:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:24:09.799 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:24:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:24:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:24:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:24:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:24:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:24:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:24:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:24:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:24:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:24:09.840 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:24:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:24:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:24:39.721 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:24:39.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:24:39.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:24:39.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:24:39.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:24:39.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:24:39.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:24:39.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:24:39.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:24:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:24:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:24:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:24:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:24:49.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:24:49.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:24:49.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:24:49.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:24:49.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:24:49.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:24:49.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:24:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:24:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:24:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:25:09.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:25:09.833 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:25:09.833 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:25:09.833 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:25:09.833 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:25:09.833 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:25:09.833 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:25:09.833 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:25:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:25:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:25:09.835 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:25:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:25:39.708 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:25:39.714 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:25:39.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:25:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:25:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:25:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:25:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:25:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:25:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:25:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:25:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:25:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:25:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:25:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:25:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:25:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:25:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:25:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:25:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:25:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:25:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:25:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:25:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:25:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:26:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:26:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:26:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:26:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:26:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:26:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:26:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:26:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:26:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:26:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:26:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:26:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:26:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:26:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:26:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:26:39.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:26:39.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:26:39.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:26:39.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:26:39.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:26:39.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:26:39.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:26:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:26:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.051 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:26:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:26:49.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:26:49.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:26:49.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:26:49.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:26:49.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:26:49.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:26:49.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:26:49.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:26:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:26:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:26:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:27:09.802 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:27:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:27:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:27:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:27:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:27:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:27:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:27:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:27:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:27:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:27:09.839 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:27:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:27:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:27:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:27:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:27:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:27:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:27:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:27:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:27:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:27:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:27:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:27:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:27:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:27:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:27:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:27:49.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:27:49.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:27:49.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:27:49.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:27:49.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:27:49.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:27:49.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:27:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:27:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:27:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:28:09.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:28:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:28:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:28:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:28:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:28:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:28:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:28:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:28:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:28:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:28:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:28:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:28:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:28:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:28:39.707 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:28:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:28:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:28:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:28:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:28:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:28:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:28:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:28:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:28:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:28:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:28:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:28:49.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:28:49.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:28:49.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:28:49.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:28:49.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:28:49.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:28:49.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:28:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:28:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:28:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:29:09.781 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:29:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:29:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:29:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:29:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:29:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:29:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:29:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:29:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:29:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:29:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:29:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:29:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:29:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:29:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:29:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:29:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:29:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:29:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:29:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:29:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:29:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:29:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:29:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:29:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:29:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:29:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:29:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:29:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:29:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:29:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:29:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:29:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:29:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:29:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:29:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:30:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:30:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:30:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:30:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:30:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:30:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:30:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:30:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:30:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:30:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:30:09.818 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:30:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:30:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:30:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:30:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:30:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:30:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:30:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:30:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:30:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:30:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:30:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:30:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:30:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:30:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:30:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:30:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:30:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:30:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:30:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:30:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:30:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:30:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:30:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:30:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:30:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:31:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:31:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:31:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:31:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:31:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:31:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:31:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:31:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:31:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:31:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:31:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:31:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:31:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:31:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:31:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:31:39.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:31:39.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:31:39.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:31:39.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:31:39.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:31:39.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:31:39.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:31:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:31:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:31:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:31:49.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:31:49.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:31:49.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:31:49.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:31:49.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:31:49.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:31:49.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:31:49.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:31:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:31:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:31:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:32:09.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:32:09.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:32:09.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:32:09.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:32:09.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:32:09.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:32:09.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:32:09.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:32:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:32:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:32:09.827 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:32:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:32:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:32:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:32:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:32:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:32:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:32:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:32:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:32:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:32:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:32:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:32:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:32:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:32:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:32:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:32:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:32:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:32:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:32:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:32:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:32:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:32:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:32:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:32:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:32:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:33:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:33:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:33:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:33:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:33:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:33:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:33:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:33:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:33:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:33:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:33:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:33:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:33:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:33:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:33:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:33:39.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:33:39.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:33:39.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:33:39.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:33:39.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:33:39.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:33:39.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:33:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:33:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.058 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:33:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:33:49.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:33:49.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:33:49.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:33:49.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:33:49.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:33:49.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:33:49.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:33:49.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:33:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:33:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.095 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:33:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:34:09.802 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:34:09.845 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:34:09.845 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:34:09.845 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:34:09.845 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:34:09.845 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:34:09.845 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:34:09.845 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:34:09.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:34:09.848 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.141 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:34:09.848 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:34:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:34:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:34:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:34:39.710 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:34:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:34:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:34:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:34:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:34:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:34:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:34:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:34:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:34:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:34:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:34:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:34:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:34:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:34:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:34:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:34:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:34:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:34:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:34:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:34:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:34:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:35:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:35:09.818 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:35:09.818 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:35:09.818 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:35:09.818 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:35:09.818 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:35:09.818 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:35:09.818 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:35:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:35:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:35:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:35:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:35:39.703 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:35:39.717 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:35:39.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:35:39.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:35:39.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:35:39.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:35:39.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:35:39.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:35:39.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:35:39.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:35:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:35:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:35:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:35:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:35:49.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:35:49.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:35:49.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:35:49.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:35:49.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:35:49.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:35:49.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:35:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:35:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:35:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:36:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:36:09.835 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:36:09.835 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:36:09.835 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:36:09.835 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:36:09.835 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:36:09.835 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:36:09.835 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:36:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:36:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:36:09.838 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:36:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:36:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:36:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:36:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:36:39.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:36:39.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:36:39.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:36:39.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:36:39.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:36:39.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:36:39.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:36:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:36:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.055 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:36:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:36:49.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:36:49.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:36:49.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:36:49.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:36:49.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:36:49.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:36:49.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:36:49.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:36:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:36:49.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.099 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:36:49.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:37:09.809 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:37:09.843 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:37:09.843 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:37:09.843 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:37:09.843 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:37:09.843 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:37:09.843 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:37:09.843 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:37:09.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:37:09.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.136 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:37:09.846 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:37:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:37:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:37:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:37:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:37:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:37:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:37:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:37:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:37:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:37:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:37:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:37:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:37:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:37:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:37:49.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:37:49.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:37:49.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:37:49.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:37:49.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:37:49.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:37:49.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:37:49.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:37:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:37:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:37:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:38:09.784 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:38:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:38:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:38:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:38:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:38:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:38:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:38:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:38:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:38:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:38:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:38:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:38:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:38:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:38:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:38:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:38:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:38:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:38:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:38:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:38:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:38:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:38:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:38:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:38:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:38:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:38:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:38:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:38:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:38:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:38:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:38:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:38:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:38:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:38:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:38:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:39:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:39:09.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:39:09.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:39:09.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:39:09.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:39:09.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:39:09.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:39:09.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:39:09.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:39:09.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:39:09.819 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:39:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:39:39.691 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:39:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:39:39.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:39:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:39:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:39:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:39:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:39:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:39:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:39:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:39:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:39:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:39:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:39:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:39:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:39:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:39:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:39:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:39:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:39:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:39:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:39:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:39:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:39:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:40:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:40:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:40:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:40:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:40:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:40:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:40:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:40:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:40:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:40:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:40:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:40:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:40:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:40:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:40:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:40:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:40:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:40:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:40:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:40:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:40:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:40:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:40:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:40:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:40:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:40:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:40:49.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:40:49.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:40:49.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:40:49.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:40:49.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:40:49.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:40:49.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:40:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:40:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:40:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:41:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:41:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:41:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:41:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:41:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:41:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:41:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:41:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:41:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:41:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:41:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:41:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:41:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:42:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:42:39.686 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:42:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:42:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:42:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:42:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:42:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:42:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:42:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:42:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:42:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:42:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:42:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:42:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:42:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:42:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:42:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:42:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:42:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:42:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:42:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:42:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:42:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:42:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:42:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:43:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:43:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:43:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:43:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:43:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:43:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:43:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:43:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:43:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:43:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:43:09.808 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:43:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:43:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:43:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:43:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:43:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:43:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:43:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:43:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:43:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:43:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:43:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:43:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:43:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:43:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:43:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:43:49.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:43:49.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:43:49.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:43:49.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:43:49.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:43:49.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:43:49.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:43:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:43:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:43:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:44:09.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:44:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:44:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:44:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:44:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:44:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:44:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:44:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:44:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:44:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:44:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:44:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:44:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:44:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:44:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:44:39.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:44:39.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:44:39.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:44:39.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:44:39.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:44:39.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:44:39.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:44:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:44:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:44:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:44:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:44:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:44:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:44:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:44:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:44:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:44:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:44:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:44:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:44:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:44:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:45:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:45:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:45:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:45:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:45:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:45:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:45:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:45:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:45:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:45:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:45:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:45:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:45:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:45:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:45:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:45:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:45:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:45:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:45:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:45:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:45:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:45:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:45:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:45:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:45:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:45:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:45:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:45:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:45:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:45:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:45:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:45:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:45:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:45:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:45:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:45:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:46:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:46:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:46:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:46:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:46:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:46:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:46:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:46:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:46:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:46:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:46:09.814 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:46:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:46:39.697 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:46:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:46:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:46:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:46:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:46:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:46:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:46:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:46:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:46:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:46:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:46:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:46:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:46:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:46:49.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:46:49.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:46:49.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:46:49.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:46:49.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:46:49.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:46:49.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:46:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:46:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:46:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:47:09.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:47:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:47:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:47:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:47:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:47:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:47:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:47:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:47:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:47:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:47:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:47:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:47:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:47:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:47:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:47:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:47:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:47:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:47:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:47:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:47:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:47:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:47:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:47:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:47:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:47:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:47:49.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:47:49.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:47:49.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:47:49.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:47:49.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:47:49.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:47:49.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:47:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:47:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:47:49.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:48:09.781 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:48:09.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:48:09.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:48:09.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:48:09.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:48:09.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:48:09.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:48:09.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:48:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:48:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:48:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:48:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:48:39.692 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:48:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:48:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:48:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:48:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:48:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:48:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:48:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:48:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:48:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:48:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:48:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:48:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:48:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:48:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:48:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:48:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:48:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:48:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:48:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:48:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:48:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:48:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:48:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:49:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:49:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:49:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:49:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:49:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:49:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:49:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:49:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:49:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:49:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:49:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:49:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:49:39.691 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:49:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:49:39.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:49:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:49:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:49:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:49:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:49:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:49:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:49:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:49:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:49:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:49:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:49:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:49:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:49:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:49:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:49:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:49:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:49:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:49:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:49:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:49:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:49:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:50:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:50:09.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:50:09.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:50:09.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:50:09.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:50:09.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:50:09.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:50:09.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:50:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:50:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:50:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:50:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:50:39.695 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:50:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:50:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:50:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:50:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:50:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:50:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:50:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:50:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:50:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:50:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:50:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.032 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:50:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:50:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:50:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:50:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:50:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:50:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:50:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:50:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:50:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:50:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:50:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:50:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:51:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:51:09.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:51:09.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:51:09.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:51:09.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:51:09.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:51:09.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:51:09.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:51:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:51:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:51:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:51:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:51:39.692 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:51:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:51:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:51:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:51:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:51:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:51:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:51:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:51:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:51:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:51:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:51:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:51:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:51:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:51:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:51:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:51:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:51:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:51:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:51:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:51:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:51:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:51:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:51:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:52:09.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:52:09.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:52:09.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:52:09.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:52:09.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:52:09.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:52:09.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:52:09.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:52:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:52:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:52:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:52:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:52:39.692 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:52:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:52:39.699 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:52:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:52:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:52:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:52:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:52:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:52:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:52:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:52:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:52:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:52:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:52:49.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:52:49.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:52:49.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:52:49.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:52:49.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:52:49.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:52:49.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:52:49.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:52:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:52:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:52:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:53:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:53:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:53:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:53:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:53:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:53:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:53:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:53:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:53:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:53:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:53:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:53:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:53:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:53:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:53:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:53:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:53:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:53:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:53:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:53:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:53:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:53:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:53:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:53:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:53:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:53:49.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:53:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:53:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:53:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:53:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:53:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:53:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:53:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:53:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:53:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:53:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:54:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:54:09.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:54:09.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:54:09.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:54:09.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:54:09.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:54:09.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:54:09.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:54:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:54:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:54:09.825 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:54:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:54:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:54:39.694 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:54:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:54:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:54:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:54:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:54:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:54:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:54:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:54:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:54:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:54:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:54:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:54:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:54:49.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:54:49.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:54:49.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:54:49.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:54:49.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:54:49.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:54:49.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:54:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:54:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:54:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:55:09.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:55:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:55:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:55:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:55:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:55:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:55:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:55:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:55:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:55:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:55:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:55:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:55:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:55:39.717 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:55:39.717 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:55:39.717 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:55:39.717 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:55:39.717 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:55:39.717 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:55:39.717 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:55:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:55:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:55:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:55:49.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:55:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:55:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:55:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:55:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:55:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:55:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:55:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:55:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:55:49.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:55:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:56:09.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:56:09.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:56:09.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:56:09.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:56:09.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:56:09.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:56:09.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:56:09.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:56:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:56:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:56:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:56:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:56:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:56:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:56:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:56:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:56:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:56:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:56:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:56:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:56:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:56:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:56:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:56:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.061 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:56:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:56:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:56:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:56:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:56:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:56:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:56:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:56:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:56:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:56:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:56:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.098 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:56:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:57:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:57:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:57:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:57:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:57:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:57:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:57:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:57:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:57:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:57:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.133 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:57:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:57:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:57:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:57:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:57:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:57:39.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:57:39.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:57:39.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:57:39.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:57:39.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:57:39.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:57:39.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:57:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:57:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:57:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:57:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:57:49.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:57:49.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:57:49.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:57:49.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:57:49.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:57:49.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:57:49.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:57:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:57:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:57:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:58:09.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:58:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:58:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:58:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:58:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:58:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:58:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:58:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:58:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:58:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:58:09.801 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:58:39.672 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:58:39.673 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:58:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:58:39.683 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:58:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:58:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:58:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:58:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:58:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:58:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:58:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:58:39.718 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:58:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:58:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:58:49.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:58:49.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:58:49.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:58:49.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:58:49.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:58:49.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:58:49.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:58:49.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:58:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:58:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:58:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:59:09.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:59:09.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:59:09.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:59:09.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:59:09.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:59:09.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:59:09.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:59:09.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:59:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:59:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 02:59:09.795 139328 ERROR oslo_service.periodic_task 
>2018-06-25 02:59:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:59:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 02:59:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 02:59:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:59:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:59:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:59:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:59:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:59:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:59:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:59:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:59:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:59:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:59:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 02:59:49.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 02:59:49.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 02:59:49.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 02:59:49.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 02:59:49.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 02:59:49.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 02:59:49.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 02:59:49.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 02:59:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 02:59:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 02:59:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:00:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:00:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:00:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:00:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:00:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:00:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:00:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:00:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:00:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:00:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:00:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:00:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:00:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:00:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:00:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:00:39.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:00:39.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:00:39.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:00:39.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:00:39.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:00:39.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:00:39.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:00:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:00:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:00:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:00:49.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:00:49.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:00:49.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:00:49.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:00:49.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:00:49.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:00:49.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:00:49.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:00:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:00:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:00:49.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:01:09.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:01:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:01:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:01:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:01:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:01:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:01:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:01:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:01:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:01:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:01:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:01:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:01:39.680 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:01:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:01:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:01:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:01:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:01:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:01:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:01:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:01:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:01:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:01:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:01:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:01:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:01:49.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:01:49.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:01:49.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:01:49.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:01:49.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:01:49.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:01:49.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:01:49.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:01:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:01:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:01:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:02:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:02:09.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:02:09.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:02:09.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:02:09.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:02:09.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:02:09.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:02:09.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:02:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:02:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:02:09.801 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:02:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:02:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:02:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:02:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:02:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:02:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:02:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:02:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:02:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:02:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:02:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:02:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:02:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:02:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:02:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:02:49.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:02:49.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:02:49.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:02:49.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:02:49.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:02:49.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:02:49.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:02:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:02:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:02:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:03:09.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:03:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:03:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:03:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:03:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:03:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:03:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:03:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:03:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:03:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:03:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:03:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:03:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:03:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:03:39.694 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:03:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:03:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:03:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:03:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:03:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:03:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:03:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:03:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:03:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:03:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:03:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:03:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:03:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:03:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:03:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:03:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:03:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:03:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:03:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:03:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:03:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:04:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:04:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:04:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:04:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:04:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:04:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:04:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:04:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:04:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:04:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:04:09.808 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:04:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:04:39.691 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:04:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:04:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:04:39.730 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:04:39.730 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:04:39.730 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:04:39.730 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:04:39.730 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:04:39.730 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:04:39.730 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:04:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:04:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:04:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:04:49.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:04:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:04:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:04:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:04:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:04:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:04:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:04:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:04:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:04:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:04:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:05:09.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:05:09.818 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:05:09.818 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:05:09.818 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:05:09.818 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:05:09.818 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:05:09.818 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:05:09.818 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:05:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:05:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:05:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:05:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:05:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:05:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:05:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:05:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:05:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:05:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:05:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:05:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:05:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:05:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:05:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:05:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:05:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:05:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:05:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:05:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:05:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:05:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:05:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:05:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:05:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:05:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:05:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:05:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:06:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:06:09.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:06:09.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:06:09.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:06:09.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:06:09.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:06:09.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:06:09.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:06:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:06:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:06:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:06:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:06:39.690 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:06:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:06:39.702 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:06:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:06:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:06:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:06:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:06:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:06:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:06:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:06:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:06:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:06:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:06:49.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:06:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:06:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:06:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:06:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:06:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:06:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:06:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:06:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:06:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:06:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:07:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:07:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:07:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:07:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:07:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:07:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:07:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:07:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:07:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:07:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:07:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:07:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:07:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:07:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:07:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:07:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:07:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:07:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:07:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:07:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:07:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:07:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:07:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:07:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:07:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:07:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:07:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:07:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:07:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:07:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:07:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:07:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:07:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:07:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:07:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:07:49.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:08:09.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:08:09.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:08:09.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:08:09.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:08:09.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:08:09.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:08:09.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:08:09.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:08:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:08:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:08:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:08:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:08:39.686 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:08:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:08:39.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:08:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:08:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:08:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:08:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:08:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:08:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:08:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:08:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:08:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:08:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:08:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:08:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:08:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:08:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:08:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:08:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:08:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:08:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:08:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:08:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:08:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:09:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:09:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:09:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:09:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:09:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:09:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:09:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:09:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:09:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:09:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:09:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:09:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:09:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:09:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:09:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:09:39.722 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:09:39.722 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:09:39.722 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:09:39.722 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:09:39.722 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:09:39.722 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:09:39.722 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:09:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:09:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:09:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:09:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:09:49.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:09:49.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:09:49.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:09:49.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:09:49.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:09:49.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:09:49.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:09:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:09:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:09:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:10:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:10:09.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:10:09.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:10:09.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:10:09.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:10:09.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:10:09.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:10:09.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:10:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:10:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:10:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:10:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:10:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:10:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:10:39.686 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:10:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:10:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:10:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:10:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:10:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:10:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:10:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:10:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:10:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:10:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:10:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:10:49.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:10:49.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:10:49.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:10:49.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:10:49.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:10:49.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:10:49.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:10:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:10:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:10:49.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:11:09.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:11:09.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:11:09.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:11:09.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:11:09.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:11:09.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:11:09.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:11:09.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:11:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:11:09.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:11:09.797 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:11:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:11:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:11:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:11:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:11:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:11:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:11:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:11:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:11:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:11:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:11:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:11:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:11:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:11:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:11:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:11:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:11:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:11:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:11:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:11:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:11:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:11:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:11:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:11:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:11:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:12:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:12:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:12:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:12:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:12:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:12:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:12:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:12:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:12:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:12:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:12:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:12:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:12:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:12:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:12:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:12:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:12:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:12:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:12:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:12:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:12:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:12:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:12:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:12:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.048 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:12:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:12:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:12:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:12:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:12:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:12:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:12:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:12:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:12:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:12:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:12:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:12:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:13:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:13:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:13:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:13:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:13:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:13:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:13:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:13:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:13:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:13:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:13:09.811 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:13:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:13:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:13:39.690 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:13:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:13:39.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:13:39.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:13:39.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:13:39.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:13:39.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:13:39.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:13:39.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:13:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:13:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:13:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:13:49.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:13:49.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:13:49.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:13:49.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:13:49.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:13:49.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:13:49.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:13:49.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:13:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:13:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:13:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:14:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:14:09.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:14:09.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:14:09.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:14:09.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:14:09.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:14:09.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:14:09.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:14:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:14:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:14:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:14:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:14:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:14:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:14:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:14:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:14:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:14:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:14:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:14:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:14:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:14:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:14:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:14:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:14:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:14:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:14:49.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:14:49.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:14:49.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:14:49.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:14:49.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:14:49.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:14:49.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:14:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:14:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:14:49.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:15:09.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:15:09.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:15:09.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:15:09.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:15:09.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:15:09.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:15:09.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:15:09.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:15:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:15:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:15:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:15:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:15:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:15:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:15:39.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:15:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:15:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:15:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:15:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:15:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:15:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:15:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:15:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:15:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:15:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:15:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:15:49.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:15:49.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:15:49.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:15:49.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:15:49.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:15:49.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:15:49.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:15:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:15:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:15:49.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:16:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:16:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:16:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:16:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:16:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:16:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:16:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:16:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:16:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:16:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:16:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:16:39.678 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:16:39.679 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:16:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:16:39.688 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:16:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:16:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:16:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:16:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:16:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:16:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:16:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:16:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:16:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:16:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:16:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:16:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:16:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:16:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:16:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:16:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:16:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:16:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:16:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:16:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.098 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:16:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:17:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:17:09.840 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:17:09.840 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:17:09.840 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:17:09.840 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:17:09.840 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:17:09.840 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:17:09.840 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:17:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:17:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.155 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:17:09.843 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:17:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:17:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:17:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:17:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:17:39.728 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:17:39.728 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:17:39.728 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:17:39.728 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:17:39.728 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:17:39.728 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:17:39.728 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:17:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:17:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.047 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:17:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:17:49.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:17:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:17:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:17:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:17:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:17:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:17:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:17:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:17:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:17:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.09 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:17:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:18:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:18:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:18:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:18:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:18:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:18:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:18:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:18:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:18:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:18:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.128 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:18:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:18:39.674 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:18:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:18:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:18:39.682 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:18:39.719 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:18:39.719 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:18:39.719 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:18:39.719 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:18:39.719 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:18:39.719 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:18:39.719 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:18:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:18:39.721 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:18:39.722 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:18:49.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:18:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:18:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:18:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:18:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:18:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:18:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:18:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:18:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:18:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:18:49.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:19:09.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:19:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:19:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:19:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:19:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:19:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:19:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:19:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:19:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:19:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.123 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:19:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:19:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:19:39.678 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:19:39.685 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:19:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:19:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:19:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:19:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:19:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:19:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:19:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:19:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:19:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:19:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:19:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:19:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:19:49.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:19:49.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:19:49.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:19:49.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:19:49.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:19:49.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:19:49.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:19:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:19:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:19:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:20:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:20:09.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:20:09.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:20:09.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:20:09.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:20:09.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:20:09.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:20:09.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:20:09.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:20:09.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:20:09.802 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:20:39.673 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:20:39.674 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:20:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:20:39.681 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:20:39.714 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:20:39.714 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:20:39.714 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:20:39.714 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:20:39.714 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:20:39.714 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:20:39.714 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:20:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:20:39.716 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:20:39.717 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:20:49.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:20:49.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:20:49.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:20:49.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:20:49.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:20:49.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:20:49.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:20:49.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:20:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:20:49.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:20:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:21:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:21:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:21:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:21:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:21:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:21:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:21:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:21:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:21:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:21:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:21:09.795 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:21:39.676 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:21:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:21:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:21:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:21:39.718 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:21:39.718 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:21:39.718 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:21:39.718 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:21:39.718 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:21:39.718 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:21:39.718 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:21:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:21:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:21:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:21:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:21:49.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:21:49.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:21:49.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:21:49.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:21:49.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:21:49.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:21:49.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:21:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:21:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:21:49.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:22:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:22:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:22:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:22:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:22:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:22:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:22:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:22:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:22:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:22:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:22:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:22:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:22:39.681 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:22:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:22:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:22:39.724 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:22:39.724 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:22:39.724 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:22:39.724 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:22:39.724 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:22:39.724 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:22:39.724 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:22:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:22:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:22:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:22:49.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:22:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:22:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:22:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:22:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:22:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:22:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:22:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:22:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:22:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:22:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:23:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:23:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:23:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:23:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:23:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:23:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:23:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:23:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:23:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:23:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:23:09.805 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:23:39.677 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:23:39.677 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:23:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:23:39.684 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:23:39.716 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:23:39.716 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:23:39.716 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:23:39.716 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:23:39.716 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:23:39.716 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:23:39.716 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:23:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:23:39.719 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:23:39.720 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:23:49.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:23:49.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:23:49.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:23:49.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:23:49.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:23:49.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:23:49.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:23:49.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:23:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:23:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:23:49.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:24:09.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:24:09.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:24:09.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:24:09.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:24:09.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:24:09.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:24:09.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:24:09.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:24:09.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:24:09.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:24:09.794 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:24:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:24:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:24:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:24:39.687 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:24:39.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:24:39.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:24:39.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:24:39.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:24:39.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:24:39.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:24:39.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:24:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:24:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:24:39.727 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:24:49.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:24:49.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:24:49.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:24:49.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:24:49.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:24:49.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:24:49.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:24:49.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:24:49.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:24:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:24:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:25:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:25:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:25:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:25:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:25:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:25:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:25:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:25:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:25:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:25:09.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:25:09.804 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:25:39.675 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:25:39.676 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:25:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:25:39.685 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:25:39.723 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:25:39.723 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:25:39.723 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:25:39.723 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:25:39.723 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:25:39.723 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:25:39.723 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:25:39.725 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:25:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:25:39.726 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:25:49.727 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:25:49.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:25:49.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:25:49.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:25:49.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:25:49.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:25:49.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:25:49.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:25:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:25:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:25:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:26:09.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:26:09.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:26:09.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:26:09.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:26:09.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:26:09.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:26:09.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:26:09.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:26:09.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:26:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:26:09.799 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:26:39.680 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:26:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:26:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:26:39.696 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:26:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:26:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:26:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:26:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:26:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:26:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:26:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:26:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:26:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:26:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:26:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:26:49.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:26:49.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:26:49.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:26:49.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:26:49.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:26:49.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:26:49.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:26:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:26:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:26:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:27:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:27:09.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:27:09.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:27:09.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:27:09.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:27:09.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:27:09.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:27:09.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:27:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:27:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:27:09.813 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:27:39.684 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:27:39.685 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:27:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:27:39.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:27:39.726 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:27:39.726 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:27:39.726 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:27:39.726 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:27:39.726 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:27:39.726 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:27:39.726 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:27:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:27:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:27:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:27:49.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:27:49.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:27:49.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:27:49.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:27:49.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:27:49.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:27:49.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:27:49.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:27:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:27:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:27:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:28:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:28:09.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:28:09.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:28:09.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:28:09.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:28:09.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:28:09.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:28:09.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:28:09.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:28:09.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:28:09.807 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:28:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:28:39.689 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:28:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:28:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:28:39.731 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:28:39.731 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:28:39.731 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:28:39.731 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:28:39.731 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:28:39.731 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:28:39.731 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:28:39.733 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:28:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:28:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:28:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:28:49.769 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:28:49.769 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:28:49.769 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:28:49.769 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:28:49.769 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:28:49.769 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:28:49.769 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:28:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:28:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:28:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:29:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:29:09.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:29:09.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:29:09.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:29:09.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:29:09.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:29:09.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:29:09.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:29:09.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:29:09.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:29:09.809 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:29:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:29:39.691 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:29:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:29:39.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:29:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:29:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:29:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:29:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:29:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:29:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:29:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:29:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:29:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:29:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:29:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:29:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:29:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:29:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:29:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:29:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:29:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:29:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:29:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:29:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:29:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:30:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:30:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:30:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:30:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:30:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:30:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:30:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:30:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:30:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:30:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:30:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:30:39.686 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:30:39.687 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:30:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:30:39.697 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:30:39.729 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:30:39.729 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:30:39.729 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:30:39.729 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:30:39.729 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:30:39.729 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:30:39.729 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:30:39.731 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:30:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:30:39.732 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:30:49.733 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:30:49.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:30:49.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:30:49.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:30:49.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:30:49.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:30:49.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:30:49.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:30:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:30:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.097 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:30:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:31:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:31:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:31:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:31:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:31:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:31:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:31:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:31:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:31:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:31:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.132 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:31:09.830 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:31:39.682 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:31:39.683 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:31:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:31:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:31:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:31:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:31:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:31:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:31:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:31:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:31:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:31:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:31:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:31:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:31:49.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:31:49.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:31:49.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:31:49.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:31:49.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:31:49.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:31:49.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:31:49.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:31:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:31:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:31:49.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:32:09.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:32:09.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:32:09.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:32:09.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:32:09.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:32:09.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:32:09.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:32:09.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:32:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:32:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:32:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:32:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:32:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:32:39.688 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:32:39.689 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:32:39.721 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:32:39.721 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:32:39.721 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:32:39.721 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:32:39.721 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:32:39.721 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:32:39.721 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:32:39.723 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:32:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:32:39.724 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:32:49.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:32:49.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:32:49.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:32:49.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:32:49.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:32:49.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:32:49.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:32:49.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:32:49.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:32:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:32:49.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:33:09.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:33:09.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:33:09.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:33:09.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:33:09.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:33:09.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:33:09.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:33:09.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:33:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:33:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:33:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:33:39.681 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:33:39.682 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:33:39.689 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:33:39.690 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:33:39.732 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:33:39.732 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:33:39.732 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:33:39.732 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:33:39.732 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:33:39.732 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:33:39.732 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:33:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:33:39.734 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:33:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:33:49.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:33:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:33:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:33:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:33:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:33:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:33:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:33:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:33:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:33:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:33:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:34:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:34:09.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:34:09.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:34:09.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:34:09.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:34:09.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:34:09.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:34:09.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:34:09.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:34:09.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:34:09.812 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:34:39.683 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:34:39.684 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:34:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:34:39.692 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:34:39.725 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:34:39.725 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:34:39.725 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:34:39.725 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:34:39.725 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:34:39.725 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:34:39.725 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:34:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:34:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:34:39.728 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:34:49.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:34:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:34:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:34:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:34:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:34:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:34:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:34:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:34:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:34:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:34:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:35:09.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:35:09.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:35:09.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:35:09.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:35:09.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:35:09.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:35:09.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:35:09.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:35:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:35:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:35:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:35:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:35:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:35:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:35:39.698 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:35:39.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:35:39.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:35:39.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:35:39.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:35:39.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:35:39.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:35:39.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:35:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:35:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:35:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:35:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:35:49.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:35:49.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:35:49.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:35:49.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:35:49.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:35:49.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:35:49.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:35:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:35:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:35:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:36:09.781 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:36:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:36:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:36:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:36:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:36:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:36:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:36:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:36:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:36:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:36:09.815 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:36:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:36:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:36:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:36:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:36:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:36:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:36:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:36:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:36:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:36:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:36:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:36:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:36:39.729 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:36:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:36:49.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:36:49.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:36:49.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:36:49.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:36:49.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:36:49.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:36:49.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:36:49.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:36:49.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:36:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:36:49.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:37:09.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:37:09.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:37:09.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:37:09.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:37:09.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:37:09.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:37:09.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:37:09.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:37:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:37:09.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:37:09.806 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:37:39.687 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:37:39.688 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:37:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:37:39.695 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:37:39.727 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:37:39.727 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:37:39.727 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:37:39.727 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:37:39.727 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:37:39.727 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:37:39.727 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:37:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:37:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:37:39.730 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:37:49.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:37:49.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:37:49.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:37:49.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:37:49.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:37:49.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:37:49.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:37:49.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:37:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:37:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:37:49.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:38:09.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:38:09.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:38:09.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:38:09.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:38:09.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:38:09.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:38:09.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:38:09.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:38:09.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:38:09.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.104 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:38:09.800 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:38:39.691 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:38:39.692 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:38:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:38:39.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:38:39.734 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:38:39.734 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:38:39.734 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:38:39.734 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:38:39.734 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:38:39.734 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:38:39.734 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:38:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:38:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:38:39.736 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:38:49.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:38:49.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:38:49.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:38:49.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:38:49.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:38:49.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:38:49.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:38:49.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:38:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:38:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:38:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:39:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:39:09.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:39:09.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:39:09.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:39:09.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:39:09.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:39:09.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:39:09.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:39:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:39:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:39:09.814 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:39:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:39:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:39:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:39:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:39:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:39:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:39:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:39:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:39:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:39:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:39:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:39:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:39:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:39:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:39:49.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:39:49.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:39:49.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:39:49.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:39:49.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:39:49.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:39:49.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:39:49.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:39:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:39:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:39:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:40:09.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:40:09.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:40:09.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:40:09.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:40:09.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:40:09.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:40:09.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:40:09.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:40:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:40:09.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:40:09.825 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:40:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:40:39.697 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:41:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:41:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:41:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:41:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:41:39.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:41:39.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:41:39.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:41:39.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:41:39.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:41:39.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:41:39.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:41:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:41:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:41:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:41:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:41:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:41:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:41:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:41:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:41:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:41:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:41:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:41:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:41:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:41:49.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:42:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:42:09.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:42:09.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:42:09.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:42:09.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:42:09.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:42:09.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:42:09.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:42:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:42:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:42:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:42:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:42:39.697 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:42:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:42:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:42:39.741 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:42:39.741 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:42:39.741 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:42:39.741 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:42:39.741 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:42:39.741 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:42:39.741 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:42:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:42:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:42:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:42:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:42:49.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:42:49.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:42:49.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:42:49.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:42:49.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:42:49.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:42:49.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:42:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:42:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:42:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:43:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:43:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:43:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:43:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:43:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:43:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:43:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:43:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:43:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:43:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:43:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:43:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:43:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:43:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:43:39.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:43:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:43:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:43:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:43:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:43:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:43:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:43:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:43:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:43:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:43:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:43:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:43:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:43:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:43:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:43:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:43:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:43:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:43:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:43:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:43:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:43:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:44:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:44:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:44:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:44:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:44:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:44:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:44:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:44:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:44:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:44:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:44:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:44:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:44:39.697 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:44:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:44:39.705 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:44:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:44:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:44:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:44:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:44:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:44:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:44:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:44:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:44:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:44:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:44:49.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:44:49.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:44:49.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:44:49.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:44:49.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:44:49.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:44:49.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:44:49.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:44:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:44:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:44:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:45:09.784 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:45:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:45:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:45:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:45:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:45:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:45:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:45:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:45:09.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:45:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:45:09.822 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:45:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:45:39.694 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:45:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:45:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:45:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:45:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:45:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:45:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:45:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:45:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:45:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:45:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:45:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:45:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:45:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:45:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:45:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:45:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:45:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:45:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:45:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:45:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:45:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:45:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:45:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:46:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:46:09.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:46:09.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:46:09.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:46:09.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:46:09.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:46:09.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:46:09.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:46:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:46:09.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:46:09.810 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:46:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:46:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:46:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:46:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:46:39.738 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:46:39.738 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:46:39.738 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:46:39.738 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:46:39.738 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:46:39.738 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:46:39.738 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:46:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:46:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:46:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:46:49.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:46:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:46:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:46:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:46:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:46:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:46:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:46:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:46:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:46:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:46:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:47:09.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:47:09.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:47:09.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:47:09.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:47:09.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:47:09.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:47:09.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:47:09.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:47:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:47:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:47:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:47:39.695 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:47:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:47:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:47:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:47:39.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:47:39.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:47:39.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:47:39.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:47:39.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:47:39.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:47:39.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:47:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:47:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:47:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:47:49.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:47:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:47:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:47:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:47:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:47:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:47:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:47:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:47:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:47:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:47:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:48:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:48:09.818 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:48:09.818 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:48:09.818 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:48:09.818 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:48:09.818 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:48:09.818 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:48:09.818 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:48:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:48:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:48:09.821 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:48:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:48:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:48:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:48:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:48:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:48:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:48:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:48:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:48:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:48:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:48:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:48:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:48:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:48:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:48:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:48:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:48:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:48:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:48:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:48:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:48:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:48:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:48:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:48:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:48:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:49:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:49:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:49:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:49:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:49:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:49:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:49:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:49:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:49:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:49:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:49:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:49:39.693 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:49:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:49:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:49:39.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:49:39.733 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:49:39.733 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:49:39.733 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:49:39.733 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:49:39.733 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:49:39.733 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:49:39.733 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:49:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:49:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:49:39.735 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:49:49.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:49:49.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:49:49.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:49:49.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:49:49.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:49:49.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:49:49.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:49:49.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:49:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:49:49.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:49:49.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:50:09.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:50:09.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:50:09.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:50:09.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:50:09.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:50:09.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:50:09.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:50:09.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:50:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:50:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:50:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:50:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:50:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:50:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:50:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:50:39.740 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:50:39.740 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:50:39.740 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:50:39.740 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:50:39.740 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:50:39.740 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:50:39.740 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:50:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:50:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:50:39.743 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:50:49.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:50:49.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:50:49.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:50:49.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:50:49.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:50:49.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:50:49.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:50:49.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:50:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:50:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:50:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:51:09.784 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:51:09.818 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:51:09.818 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:51:09.818 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:51:09.818 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:51:09.818 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:51:09.818 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:51:09.818 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:51:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:51:09.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:51:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:51:39.692 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:51:39.693 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:51:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:51:39.700 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:51:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:51:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:51:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:51:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:51:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:51:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:51:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:51:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:51:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:51:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:51:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:51:49.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:51:49.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:51:49.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:51:49.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:51:49.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:51:49.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:51:49.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:51:49.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:51:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:51:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:52:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:52:09.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:52:09.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:52:09.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:52:09.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:52:09.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:52:09.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:52:09.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:52:09.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:52:09.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:52:09.814 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:52:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:52:39.696 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:52:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:52:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:52:39.736 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:52:39.736 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:52:39.736 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:52:39.736 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:52:39.736 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:52:39.736 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:52:39.736 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:52:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:52:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:52:39.739 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:52:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:52:49.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:52:49.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:52:49.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:52:49.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:52:49.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:52:49.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:52:49.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:52:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:52:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:52:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:53:09.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:53:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:53:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:53:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:53:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:53:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:53:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:53:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:53:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:53:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:53:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:53:39.697 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:53:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:53:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:53:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:53:39.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:53:39.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:53:39.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:53:39.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:53:39.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:53:39.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:53:39.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:53:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:53:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:53:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:53:49.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:53:49.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:53:49.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:53:49.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:53:49.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:53:49.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:53:49.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:53:49.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:53:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:53:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:53:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:54:09.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:54:09.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:54:09.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:54:09.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:54:09.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:54:09.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:54:09.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:54:09.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:54:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:54:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:54:09.825 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:54:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:54:39.697 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:54:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:54:39.704 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:54:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:54:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:54:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:54:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:54:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:54:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:54:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:54:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:54:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:54:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:54:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:54:49.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:54:49.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:54:49.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:54:49.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:54:49.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:54:49.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:54:49.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:54:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:54:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:54:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:55:09.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:55:09.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:55:09.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:55:09.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:55:09.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:55:09.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:55:09.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:55:09.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:55:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:55:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:55:09.825 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:55:39.696 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:55:39.697 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:55:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:55:39.703 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:55:39.737 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:55:39.737 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:55:39.737 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:55:39.737 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:55:39.737 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:55:39.737 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:55:39.737 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:55:39.740 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:55:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:55:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:55:49.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:55:49.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:55:49.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:55:49.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:55:49.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:55:49.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:55:49.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:55:49.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:55:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:55:49.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:55:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:56:09.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:56:09.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:56:09.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:56:09.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:56:09.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:56:09.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:56:09.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:56:09.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:56:09.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:56:09.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:56:09.818 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:56:39.699 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:56:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:56:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:56:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:56:39.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:56:39.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:56:39.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:56:39.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:56:39.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:56:39.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:56:39.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:56:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:56:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:56:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:56:49.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:56:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:56:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:56:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:56:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:56:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:56:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:56:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:56:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:56:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.086 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:56:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:57:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:57:09.829 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:57:09.829 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:57:09.829 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:57:09.829 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:57:09.829 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:57:09.829 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:57:09.829 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:57:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:57:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:57:09.832 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:57:39.703 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:57:39.704 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:57:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:57:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:57:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:57:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:57:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:57:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:57:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:57:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:57:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:57:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:57:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:57:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:57:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:57:49.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:57:49.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:57:49.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:57:49.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:57:49.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:57:49.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:57:49.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:57:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:57:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:57:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:58:09.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:58:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:58:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:58:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:58:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:58:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:58:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:58:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:58:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:58:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:58:09.830 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:58:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:58:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:58:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:58:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:58:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:58:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:58:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:58:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:58:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:58:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:58:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:58:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:58:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:58:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:58:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:58:49.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:58:49.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:58:49.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:58:49.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:58:49.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:58:49.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:58:49.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:58:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:58:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:58:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:59:09.784 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:59:09.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:59:09.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:59:09.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:59:09.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:59:09.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:59:09.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:59:09.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:59:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:59:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 03:59:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-25 03:59:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:59:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 03:59:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 03:59:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:59:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:59:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:59:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:59:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:59:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:59:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:59:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:59:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:59:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:59:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 03:59:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 03:59:49.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 03:59:49.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 03:59:49.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 03:59:49.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 03:59:49.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 03:59:49.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 03:59:49.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 03:59:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 03:59:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 03:59:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:00:09.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:00:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:00:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:00:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:00:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:00:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:00:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:00:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:00:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:00:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:00:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:00:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:00:39.701 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:00:39.735 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:00:39.735 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:00:39.735 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:00:39.735 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:00:39.735 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:00:39.735 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:00:39.735 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:00:39.737 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:00:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:00:39.738 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:00:49.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:00:49.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:00:49.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:00:49.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:00:49.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:00:49.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:00:49.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:00:49.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:00:49.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:00:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:00:49.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:01:09.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:01:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:01:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:01:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:01:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:01:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:01:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:01:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:01:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:01:09.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:01:09.817 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:01:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:01:39.699 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:01:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:01:39.706 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:01:39.739 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:01:39.739 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:01:39.739 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:01:39.739 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:01:39.739 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:01:39.739 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:01:39.739 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:01:39.741 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:01:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:01:39.742 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:01:49.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:01:49.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:01:49.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:01:49.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:01:49.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:01:49.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:01:49.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:01:49.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:01:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:01:49.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:01:49.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:02:09.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:02:09.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:02:09.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:02:09.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:02:09.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:02:09.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:02:09.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:02:09.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:02:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:02:09.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:02:09.816 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:02:39.698 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:02:39.698 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:02:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:02:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:02:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:02:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:02:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:02:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:02:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:02:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:02:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:02:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:02:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:02:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:02:49.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:02:49.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:02:49.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:02:49.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:02:49.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:02:49.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:02:49.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:02:49.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:02:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:02:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:02:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:03:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:03:09.838 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:03:09.838 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:03:09.838 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:03:09.838 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:03:09.838 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:03:09.838 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:03:09.838 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:03:09.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:03:09.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:03:09.841 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:03:39.702 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:03:39.703 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:03:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:03:39.711 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:03:39.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:03:39.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:03:39.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:03:39.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:03:39.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:03:39.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:03:39.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:03:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:03:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:03:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:03:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:03:49.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:03:49.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:03:49.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:03:49.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:03:49.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:03:49.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:03:49.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:03:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:03:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:03:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:04:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:04:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:04:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:04:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:04:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:04:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:04:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:04:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:04:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:04:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:04:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:04:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:04:39.701 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:04:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:04:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:04:39.742 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:04:39.742 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:04:39.742 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:04:39.742 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:04:39.742 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:04:39.742 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:04:39.742 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:04:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:04:39.744 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:04:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:04:49.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:04:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:04:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:04:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:04:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:04:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:04:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:04:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:04:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:04:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:04:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:05:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:05:09.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:05:09.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:05:09.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:05:09.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:05:09.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:05:09.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:05:09.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:05:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:05:09.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:05:09.820 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:05:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:05:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:05:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:05:39.708 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:05:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:05:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:05:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:05:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:05:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:05:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:05:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:05:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:05:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:05:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:05:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:05:49.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:05:49.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:05:49.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:05:49.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:05:49.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:05:49.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:05:49.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:05:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:05:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:05:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:06:09.784 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:06:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:06:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:06:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:06:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:06:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:06:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:06:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:06:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:06:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:06:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:06:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:06:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:06:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:06:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:06:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:06:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:06:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:06:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:06:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:06:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:06:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:06:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:06:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:06:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:06:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:06:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:06:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:06:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:06:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:06:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:06:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:06:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:06:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:06:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:06:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:07:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:07:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:07:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:07:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:07:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:07:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:07:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:07:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:07:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:07:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:07:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:07:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:07:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:07:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:07:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:07:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:07:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:07:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:07:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:07:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:07:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:07:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:07:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:07:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:07:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:07:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:07:49.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:07:49.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:07:49.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:07:49.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:07:49.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:07:49.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:07:49.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:07:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:07:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:07:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:08:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:08:09.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:08:09.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:08:09.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:08:09.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:08:09.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:08:09.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:08:09.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:08:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:08:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:08:09.833 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:08:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:08:39.705 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:08:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:08:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:08:39.745 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:08:39.745 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:08:39.745 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:08:39.745 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:08:39.745 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:08:39.745 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:08:39.745 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:08:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:08:39.747 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:08:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:08:49.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:08:49.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:08:49.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:08:49.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:08:49.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:08:49.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:08:49.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:08:49.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:08:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:08:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:08:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:09:09.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:09:09.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:09:09.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:09:09.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:09:09.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:09:09.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:09:09.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:09:09.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:09:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:09:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:09:09.834 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:09:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:09:39.707 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:09:39.714 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:09:39.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:09:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:09:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:09:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:09:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:09:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:09:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:09:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:09:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:09:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:09:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:09:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:09:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:09:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:09:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:09:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:09:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:09:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:09:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:09:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:09:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:09:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:10:09.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:10:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:10:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:10:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:10:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:10:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:10:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:10:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:10:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:10:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:10:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:10:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:10:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:10:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:10:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:10:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:10:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:10:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:10:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:10:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:10:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:10:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:10:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:10:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:10:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:10:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:10:49.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:10:49.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:10:49.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:10:49.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:10:49.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:10:49.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:10:49.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:10:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:10:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:10:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:11:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:11:09.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:11:09.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:11:09.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:11:09.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:11:09.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:11:09.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:11:09.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:11:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:11:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:11:09.834 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:11:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:11:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:11:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:11:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:11:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:11:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:11:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:11:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:11:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:11:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:11:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:11:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:11:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:11:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:11:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:11:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:11:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:11:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:11:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:11:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:11:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:11:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:11:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:11:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:11:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:12:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:12:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:12:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:12:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:12:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:12:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:12:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:12:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:12:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:12:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:12:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:12:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:12:39.707 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:12:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:12:39.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:12:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:12:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:12:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:12:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:12:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:12:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:12:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:12:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:12:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:12:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:12:49.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:12:49.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:12:49.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:12:49.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:12:49.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:12:49.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:12:49.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:12:49.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:12:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:12:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:12:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:13:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:13:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:13:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:13:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:13:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:13:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:13:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:13:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:13:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:13:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:13:09.840 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:13:39.701 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:13:39.702 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:13:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:13:39.709 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:13:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:13:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:13:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:13:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:13:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:13:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:13:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:13:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:13:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:13:39.746 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:13:49.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:13:49.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:13:49.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:13:49.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:13:49.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:13:49.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:13:49.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:13:49.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:13:49.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:13:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:13:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:14:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:14:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:14:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:14:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:14:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:14:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:14:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:14:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:14:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:14:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:14:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:14:39.700 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:14:39.700 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:14:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:14:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:14:39.743 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:14:39.743 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:14:39.743 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:14:39.743 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:14:39.743 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:14:39.743 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:14:39.743 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:14:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:14:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:14:39.745 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:14:49.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:14:49.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:14:49.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:14:49.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:14:49.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:14:49.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:14:49.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:14:49.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:14:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:14:49.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:14:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:15:09.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:15:09.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:15:09.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:15:09.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:15:09.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:15:09.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:15:09.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:15:09.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:15:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:15:09.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:15:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:15:39.704 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:15:39.705 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:15:39.714 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:15:39.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:15:39.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:15:39.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:15:39.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:15:39.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:15:39.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:15:39.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:15:39.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:15:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:15:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:15:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:15:49.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:15:49.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:15:49.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:15:49.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:15:49.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:15:49.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:15:49.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:15:49.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:15:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:15:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:15:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:16:09.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:16:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:16:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:16:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:16:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:16:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:16:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:16:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:16:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:16:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:16:09.823 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:16:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:16:39.705 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:16:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:16:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:16:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:16:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:16:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:16:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:16:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:16:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:16:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:16:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:16:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:16:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:16:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:16:49.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:16:49.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:16:49.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:16:49.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:16:49.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:16:49.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:16:49.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:16:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:16:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:16:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:17:09.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:17:09.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:17:09.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:17:09.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:17:09.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:17:09.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:17:09.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:17:09.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:17:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:17:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:17:09.827 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:17:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:17:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:17:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:17:39.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:17:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:17:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:17:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:17:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:17:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:17:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:17:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:17:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:17:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:17:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:17:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:17:49.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:17:49.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:17:49.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:17:49.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:17:49.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:17:49.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:17:49.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:17:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:17:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:17:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:18:09.793 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:18:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:18:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:18:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:18:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:18:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:18:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:18:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:18:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:18:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:18:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:18:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:18:39.711 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:18:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:18:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:18:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:18:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:18:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:18:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:18:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:18:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:18:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:18:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:18:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:18:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:18:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:18:49.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:18:49.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:18:49.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:18:49.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:18:49.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:18:49.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:18:49.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:18:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:18:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:18:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:19:09.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:19:09.833 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:19:09.833 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:19:09.833 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:19:09.833 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:19:09.833 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:19:09.833 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:19:09.833 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:19:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:19:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:19:09.836 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:19:39.707 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:19:39.708 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:19:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:19:39.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:19:39.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:19:39.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:19:39.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:19:39.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:19:39.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:19:39.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:19:39.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:19:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:19:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:19:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:19:49.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:19:49.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:19:49.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:19:49.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:19:49.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:19:49.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:19:49.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:19:49.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:19:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:19:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:19:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:20:09.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:20:09.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:20:09.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:20:09.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:20:09.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:20:09.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:20:09.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:20:09.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:20:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:20:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:20:09.834 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:20:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:20:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:20:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:20:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:20:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:20:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:20:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:20:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:20:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:20:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:20:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:20:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:20:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:20:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:20:49.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:20:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:20:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:20:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:20:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:20:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:20:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:20:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:20:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:20:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:20:49.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:21:09.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:21:09.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:21:09.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:21:09.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:21:09.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:21:09.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:21:09.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:21:09.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:21:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:21:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:21:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:21:39.705 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:21:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:21:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:21:39.713 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:21:39.746 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:21:39.746 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:21:39.746 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:21:39.746 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:21:39.746 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:21:39.746 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:21:39.746 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:21:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:21:39.748 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:21:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:21:49.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:21:49.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:21:49.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:21:49.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:21:49.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:21:49.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:21:49.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:21:49.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:21:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:21:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:21:49.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:22:09.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:22:09.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:22:09.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:22:09.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:22:09.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:22:09.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:22:09.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:22:09.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:22:09.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:22:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:22:09.824 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:22:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:22:39.707 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:22:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:22:39.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:22:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:22:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:22:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:22:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:22:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:22:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:22:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:22:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:22:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:22:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:22:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:22:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:22:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:22:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:22:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:22:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:22:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:22:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:22:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:22:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:22:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:23:09.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:23:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:23:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:23:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:23:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:23:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:23:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:23:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:23:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:23:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:23:09.838 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:23:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:23:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:23:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:23:39.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:23:39.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:23:39.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:23:39.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:23:39.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:23:39.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:23:39.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:23:39.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:23:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:23:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.032 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:23:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:23:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:23:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:23:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:23:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:23:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:23:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:23:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:23:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:23:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:23:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:23:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:24:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:24:09.829 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:24:09.829 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:24:09.829 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:24:09.829 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:24:09.829 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:24:09.829 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:24:09.829 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:24:09.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:24:09.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:24:09.832 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:24:39.714 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:24:39.714 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:24:39.722 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:24:39.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:24:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:24:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:24:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:24:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:24:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:24:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:24:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:24:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:24:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:24:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:24:49.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:24:49.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:24:49.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:24:49.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:24:49.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:24:49.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:24:49.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:24:49.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:24:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:24:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:24:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:25:09.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:25:09.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:25:09.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:25:09.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:25:09.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:25:09.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:25:09.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:25:09.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:25:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:25:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:25:09.835 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:25:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:25:39.717 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:25:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:25:39.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:25:39.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:25:39.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:25:39.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:25:39.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:25:39.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:25:39.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:25:39.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:25:39.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:25:39.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:25:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:25:49.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:25:49.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:25:49.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:25:49.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:25:49.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:25:49.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:25:49.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:25:49.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:25:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:25:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:25:49.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:26:09.805 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:26:09.839 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:26:09.839 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:26:09.839 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:26:09.839 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:26:09.839 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:26:09.839 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:26:09.839 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:26:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:26:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:26:09.843 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:26:39.715 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:26:39.716 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:26:39.722 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:26:39.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:26:39.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:26:39.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:26:39.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:26:39.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:26:39.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:26:39.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:26:39.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:26:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:26:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:26:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:26:49.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:26:49.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:26:49.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:26:49.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:26:49.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:26:49.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:26:49.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:26:49.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:26:49.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:26:49.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:26:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:27:09.804 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:27:09.838 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:27:09.838 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:27:09.838 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:27:09.838 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:27:09.838 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:27:09.838 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:27:09.838 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:27:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:27:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:27:09.841 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:27:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:27:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:27:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:27:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:27:39.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:27:39.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:27:39.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:27:39.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:27:39.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:27:39.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:27:39.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:27:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:27:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:27:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:27:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:27:49.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:27:49.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:27:49.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:27:49.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:27:49.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:27:49.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:27:49.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:27:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:27:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:27:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:28:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:28:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:28:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:28:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:28:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:28:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:28:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:28:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:28:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:28:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:28:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:28:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:28:39.711 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:28:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:28:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:28:39.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:28:39.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:28:39.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:28:39.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:28:39.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:28:39.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:28:39.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:28:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:28:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:28:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:28:49.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:28:49.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:28:49.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:28:49.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:28:49.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:28:49.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:28:49.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:28:49.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:28:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:28:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:28:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:29:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:29:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:29:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:29:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:29:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:29:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:29:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:29:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:29:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:29:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:29:09.830 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:29:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:29:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:29:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:29:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:29:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:29:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:29:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:29:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:29:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:29:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:29:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:29:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:29:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:29:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:29:49.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:29:49.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:29:49.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:29:49.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:29:49.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:29:49.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:29:49.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:29:49.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:29:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:29:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:29:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:30:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:30:09.829 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:30:09.829 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:30:09.829 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:30:09.829 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:30:09.829 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:30:09.829 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:30:09.829 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:30:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:30:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:30:09.832 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:30:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:30:39.714 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:30:39.727 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:30:39.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:30:39.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:30:39.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:30:39.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:30:39.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:30:39.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:30:39.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:30:39.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:30:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:30:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:30:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:30:49.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:30:49.806 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:30:49.806 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:30:49.806 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:30:49.806 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:30:49.806 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:30:49.806 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:30:49.806 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:30:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:30:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:30:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:31:09.809 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:31:09.844 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:31:09.844 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:31:09.844 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:31:09.844 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:31:09.844 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:31:09.844 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:31:09.844 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:31:09.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:31:09.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:31:09.846 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:31:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:31:39.718 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:31:39.725 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:31:39.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:31:39.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:31:39.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:31:39.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:31:39.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:31:39.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:31:39.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:31:39.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:31:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:31:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:31:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:31:49.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:31:49.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:31:49.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:31:49.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:31:49.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:31:49.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:31:49.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:31:49.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:31:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:31:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:31:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:32:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:32:09.839 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:32:09.839 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:32:09.839 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:32:09.839 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:32:09.839 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:32:09.839 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:32:09.839 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:32:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:32:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:32:09.842 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:32:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:32:39.714 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:32:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:32:39.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:32:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:32:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:32:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:32:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:32:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:32:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:32:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:32:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:32:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:32:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:32:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:32:49.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:32:49.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:32:49.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:32:49.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:32:49.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:32:49.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:32:49.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:32:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:32:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:32:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:33:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:33:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:33:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:33:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:33:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:33:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:33:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:33:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:33:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:33:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:33:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:33:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:33:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:33:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:33:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:33:39.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:33:39.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:33:39.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:33:39.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:33:39.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:33:39.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:33:39.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:33:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:33:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:33:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:33:49.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:33:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:33:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:33:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:33:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:33:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:33:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:33:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:33:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:33:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:33:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:34:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:34:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:34:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:34:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:34:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:34:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:34:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:34:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:34:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:34:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:34:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:34:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:34:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:34:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:34:39.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:34:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:34:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:34:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:34:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:34:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:34:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:34:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:34:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:34:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:34:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:34:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:34:49.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:34:49.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:34:49.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:34:49.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:34:49.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:34:49.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:34:49.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:34:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:34:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:34:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:35:09.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:35:09.832 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:35:09.832 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:35:09.832 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:35:09.832 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:35:09.832 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:35:09.832 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:35:09.832 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:35:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:35:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:35:09.836 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:35:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:35:39.719 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:35:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:35:39.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:35:39.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:35:39.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:35:39.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:35:39.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:35:39.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:35:39.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:35:39.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:35:39.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:35:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:35:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:35:49.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:35:49.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:35:49.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:35:49.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:35:49.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:35:49.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:35:49.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:35:49.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:35:49.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:35:49.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:35:49.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:36:09.808 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:36:09.842 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:36:09.842 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:36:09.842 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:36:09.842 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:36:09.842 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:36:09.842 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:36:09.842 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:36:09.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:36:09.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:36:09.845 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:36:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:36:39.717 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:36:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:36:39.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:36:39.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:36:39.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:36:39.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:36:39.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:36:39.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:36:39.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:36:39.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:36:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:36:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:36:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:36:49.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:36:49.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:36:49.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:36:49.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:36:49.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:36:49.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:36:49.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:36:49.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:36:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:36:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:36:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:37:09.802 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:37:09.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:37:09.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:37:09.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:37:09.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:37:09.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:37:09.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:37:09.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:37:09.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:37:09.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:37:09.841 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:37:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:37:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:37:39.721 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:37:39.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:37:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:37:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:37:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:37:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:37:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:37:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:37:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:37:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:37:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:37:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:37:49.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:37:49.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:37:49.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:37:49.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:37:49.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:37:49.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:37:49.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:37:49.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:37:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:37:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:37:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:38:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:38:09.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:38:09.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:38:09.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:38:09.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:38:09.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:38:09.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:38:09.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:38:09.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:38:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:38:09.841 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:38:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:38:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:38:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:38:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:38:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:38:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:38:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:38:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:38:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:38:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:38:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:38:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:38:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:38:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:38:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:38:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:38:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:38:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:38:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:38:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:38:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:38:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:38:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:38:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:38:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:39:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:39:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:39:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:39:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:39:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:39:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:39:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:39:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:39:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:39:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:39:09.830 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:39:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:39:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:39:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:39:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:39:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:39:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:39:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:39:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:39:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:39:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:39:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:39:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:39:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:39:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:39:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:39:49.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:39:49.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:39:49.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:39:49.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:39:49.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:39:49.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:39:49.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:39:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:39:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:39:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:40:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:40:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:40:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:40:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:40:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:40:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:40:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:40:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:40:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:40:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:40:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:40:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:40:39.714 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:40:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:40:39.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:40:39.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:40:39.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:40:39.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:40:39.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:40:39.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:40:39.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:40:39.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:40:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:40:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:40:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:40:49.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:40:49.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:40:49.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:40:49.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:40:49.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:40:49.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:40:49.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:40:49.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:40:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:40:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:40:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:41:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:41:09.839 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:41:09.839 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:41:09.839 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:41:09.839 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:41:09.839 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:41:09.839 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:41:09.839 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:41:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:41:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:41:09.841 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:41:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:41:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:42:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:42:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:42:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:42:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:42:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:42:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:42:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:42:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:42:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:42:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:42:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:42:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:42:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:42:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:42:49.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:42:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:42:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:42:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:42:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:42:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:42:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:42:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:42:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:42:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:42:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:43:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:43:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:43:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:43:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:43:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:43:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:43:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:43:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:43:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:43:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:43:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:43:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:43:39.714 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:43:39.721 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:43:39.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:43:39.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:43:39.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:43:39.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:43:39.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:43:39.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:43:39.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:43:39.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:43:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:43:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:43:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:43:49.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:43:49.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:43:49.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:43:49.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:43:49.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:43:49.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:43:49.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:43:49.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:43:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:43:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:43:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:44:09.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:44:09.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:44:09.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:44:09.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:44:09.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:44:09.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:44:09.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:44:09.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:44:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:44:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:44:09.834 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:44:39.715 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:44:39.716 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:44:39.722 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:44:39.723 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:44:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:44:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:44:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:44:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:44:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:44:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:44:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:44:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:44:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:44:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:44:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:44:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:44:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:44:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:44:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:44:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:44:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:44:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:44:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:44:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:44:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:45:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:45:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:45:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:45:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:45:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:45:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:45:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:45:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:45:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:45:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:45:09.832 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:45:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:45:39.714 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:45:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:45:39.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:45:39.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:45:39.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:45:39.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:45:39.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:45:39.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:45:39.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:45:39.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:45:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:45:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:45:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:45:49.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:45:49.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:45:49.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:45:49.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:45:49.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:45:49.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:45:49.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:45:49.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:45:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:45:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:45:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:46:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:46:09.848 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:46:09.848 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:46:09.848 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:46:09.848 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:46:09.848 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:46:09.848 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:46:09.848 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:46:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:46:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:46:09.851 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:46:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:46:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:46:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:46:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:46:39.754 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:46:39.754 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:46:39.754 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:46:39.754 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:46:39.754 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:46:39.754 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:46:39.754 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:46:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:46:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:46:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:46:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:46:49.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:46:49.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:46:49.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:46:49.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:46:49.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:46:49.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:46:49.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:46:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:46:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:46:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:47:09.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:47:09.829 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:47:09.829 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:47:09.829 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:47:09.829 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:47:09.829 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:47:09.829 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:47:09.829 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:47:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:47:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:47:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:47:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:47:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:47:39.721 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:47:39.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:47:39.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:47:39.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:47:39.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:47:39.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:47:39.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:47:39.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:47:39.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:47:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:47:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:47:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:47:49.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:47:49.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:47:49.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:47:49.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:47:49.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:47:49.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:47:49.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:47:49.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:47:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:47:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:47:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:48:09.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:48:09.834 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:48:09.834 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:48:09.834 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:48:09.834 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:48:09.834 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:48:09.834 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:48:09.834 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:48:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:48:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:48:09.837 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:48:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:48:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:48:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:48:39.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:48:39.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:48:39.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:48:39.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:48:39.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:48:39.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:48:39.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:48:39.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:48:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:48:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:48:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:48:49.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:48:49.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:48:49.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:48:49.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:48:49.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:48:49.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:48:49.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:48:49.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:48:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:48:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:48:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:49:09.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:49:09.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:49:09.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:49:09.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:49:09.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:49:09.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:49:09.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:49:09.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:49:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:49:09.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:49:09.825 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:49:39.706 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:49:39.706 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:49:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:49:39.714 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:49:39.748 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:49:39.748 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:49:39.748 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:49:39.748 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:49:39.748 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:49:39.748 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:49:39.748 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:49:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:49:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:49:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:49:49.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:49:49.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:49:49.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:49:49.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:49:49.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:49:49.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:49:49.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:49:49.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:49:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:49:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:49:49.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:50:09.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:50:09.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:50:09.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:50:09.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:50:09.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:50:09.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:50:09.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:50:09.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:50:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:50:09.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:50:09.826 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:50:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:50:39.708 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:50:39.715 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:50:39.715 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:50:39.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:50:39.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:50:39.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:50:39.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:50:39.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:50:39.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:50:39.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:50:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:50:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:50:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:50:49.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:50:49.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:50:49.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:50:49.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:50:49.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:50:49.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:50:49.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:50:49.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:50:49.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:50:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:50:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:51:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:51:09.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:51:09.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:51:09.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:51:09.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:51:09.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:51:09.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:51:09.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:51:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:51:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:51:09.830 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:51:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:51:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:51:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:51:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:51:39.750 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:51:39.750 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:51:39.750 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:51:39.750 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:51:39.750 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:51:39.750 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:51:39.750 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:51:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:51:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:51:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:51:49.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:51:49.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:51:49.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:51:49.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:51:49.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:51:49.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:51:49.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:51:49.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:51:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:51:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:51:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:52:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:52:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:52:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:52:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:52:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:52:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:52:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:52:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:52:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:52:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:52:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:52:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:52:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:52:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:52:39.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:52:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:52:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:52:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:52:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:52:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:52:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:52:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:52:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:52:39.754 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:52:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:52:49.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:52:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:52:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:52:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:52:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:52:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:52:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:52:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:52:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:52:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:52:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:53:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:53:09.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:53:09.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:53:09.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:53:09.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:53:09.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:53:09.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:53:09.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:53:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:53:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:53:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:53:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:53:39.711 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:53:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:53:39.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:53:39.753 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:53:39.753 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:53:39.753 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:53:39.753 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:53:39.753 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:53:39.753 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:53:39.753 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:53:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:53:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:53:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:53:49.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:53:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:53:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:53:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:53:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:53:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:53:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:53:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:53:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:53:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:53:49.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:54:09.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:54:09.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:54:09.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:54:09.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:54:09.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:54:09.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:54:09.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:54:09.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:54:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:54:09.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:54:09.829 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:54:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:54:39.711 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:54:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:54:39.721 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:54:39.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:54:39.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:54:39.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:54:39.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:54:39.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:54:39.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:54:39.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:54:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:54:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:54:39.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:54:49.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:54:49.801 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:54:49.801 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:54:49.801 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:54:49.801 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:54:49.801 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:54:49.801 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:54:49.801 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:54:49.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:54:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:54:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:55:09.805 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:55:09.839 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:55:09.839 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:55:09.839 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:55:09.839 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:55:09.839 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:55:09.839 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:55:09.839 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:55:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:55:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:55:09.842 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:55:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:55:39.714 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:55:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:55:39.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:55:39.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:55:39.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:55:39.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:55:39.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:55:39.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:55:39.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:55:39.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:55:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:55:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:55:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:55:49.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:55:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:55:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:55:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:55:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:55:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:55:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:55:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:55:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:55:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.092 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:55:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:56:09.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:56:09.851 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:56:09.851 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:56:09.851 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:56:09.851 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:56:09.851 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:56:09.851 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:56:09.851 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:56:09.854 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:56:09.854 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:56:09.854 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:56:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:56:39.717 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:56:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:56:39.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:56:39.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:56:39.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:56:39.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:56:39.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:56:39.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:56:39.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:56:39.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:56:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:56:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:56:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:56:49.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:56:49.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:56:49.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:56:49.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:56:49.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:56:49.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:56:49.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:56:49.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:56:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:56:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:56:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:57:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:57:09.849 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:57:09.849 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:57:09.849 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:57:09.849 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:57:09.849 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:57:09.849 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:57:09.849 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:57:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:57:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:57:09.852 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:57:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:57:39.714 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:57:39.721 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:57:39.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:57:39.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:57:39.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:57:39.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:57:39.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:57:39.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:57:39.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:57:39.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:57:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:57:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:57:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:57:49.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:57:49.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:57:49.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:57:49.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:57:49.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:57:49.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:57:49.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:57:49.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:57:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:57:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.096 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:57:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:58:09.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:58:09.852 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:58:09.852 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:58:09.852 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:58:09.852 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:58:09.852 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:58:09.852 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:58:09.852 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:58:09.854 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:58:09.855 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.133 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:58:09.855 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:58:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:58:39.717 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:58:39.723 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:58:39.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:58:39.756 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:58:39.756 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:58:39.756 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:58:39.756 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:58:39.756 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:58:39.756 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:58:39.756 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:58:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:58:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:58:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:58:49.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:58:49.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:58:49.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:58:49.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:58:49.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:58:49.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:58:49.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:58:49.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:58:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:58:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:58:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:59:09.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:59:09.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:59:09.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:59:09.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:59:09.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:59:09.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:59:09.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:59:09.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:59:09.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:59:09.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 04:59:09.833 139328 ERROR oslo_service.periodic_task 
>2018-06-25 04:59:39.715 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:59:39.715 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 04:59:39.721 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 04:59:39.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:59:39.755 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:59:39.755 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:59:39.755 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:59:39.755 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:59:39.755 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:59:39.755 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:59:39.755 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:59:39.757 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:59:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:59:39.758 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 04:59:49.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 04:59:49.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 04:59:49.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 04:59:49.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 04:59:49.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 04:59:49.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 04:59:49.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 04:59:49.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 04:59:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 04:59:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 04:59:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:00:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:00:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:00:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:00:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:00:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:00:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:00:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:00:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:00:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:00:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:00:09.839 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:00:39.711 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:00:39.712 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:00:39.747 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:00:39.747 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:00:39.747 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:00:39.747 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:00:39.747 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:00:39.747 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:00:39.747 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:00:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:00:39.749 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:00:39.750 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:00:49.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:00:49.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:00:49.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:00:49.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:00:49.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:00:49.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:00:49.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:00:49.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:00:49.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:00:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:00:49.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:01:09.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:01:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:01:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:01:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:01:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:01:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:01:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:01:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:01:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:01:09.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:01:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:01:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:01:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:01:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:01:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:01:39.752 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:01:39.752 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:01:39.752 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:01:39.752 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:01:39.752 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:01:39.752 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:01:39.752 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:01:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:01:39.755 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:01:39.756 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:01:49.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:01:49.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:01:49.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:01:49.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:01:49.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:01:49.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:01:49.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:01:49.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:01:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:01:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:01:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:02:09.800 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:02:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:02:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:02:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:02:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:02:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:02:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:02:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:02:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:02:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:02:09.839 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:02:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:02:39.711 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:02:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:02:39.719 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:02:39.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:02:39.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:02:39.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:02:39.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:02:39.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:02:39.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:02:39.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:02:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:02:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:02:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:02:49.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:02:49.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:02:49.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:02:49.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:02:49.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:02:49.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:02:49.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:02:49.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:02:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:02:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:02:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:03:09.803 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:03:09.836 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:03:09.836 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:03:09.836 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:03:09.836 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:03:09.836 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:03:09.836 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:03:09.836 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:03:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:03:09.838 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:03:09.838 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:03:39.710 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:03:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:03:39.723 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:03:39.724 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:03:39.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:03:39.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:03:39.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:03:39.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:03:39.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:03:39.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:03:39.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:03:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:03:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:03:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:03:49.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:03:49.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:03:49.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:03:49.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:03:49.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:03:49.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:03:49.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:03:49.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:03:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:03:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:03:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:04:09.802 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:04:09.835 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:04:09.835 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:04:09.835 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:04:09.835 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:04:09.835 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:04:09.835 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:04:09.835 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:04:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:04:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:04:09.838 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:04:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:04:39.709 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:04:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:04:39.716 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:04:39.749 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:04:39.749 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:04:39.749 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:04:39.749 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:04:39.749 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:04:39.749 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:04:39.749 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:04:39.751 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:04:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:04:39.752 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:04:49.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:04:49.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:04:49.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:04:49.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:04:49.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:04:49.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:04:49.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:04:49.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:04:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:04:49.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:04:49.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:05:09.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:05:09.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:05:09.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:05:09.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:05:09.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:05:09.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:05:09.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:05:09.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:05:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:05:09.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:05:09.828 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:05:39.709 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:05:39.710 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:05:39.717 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:05:39.717 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:05:39.751 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:05:39.751 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:05:39.751 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:05:39.751 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:05:39.751 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:05:39.751 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:05:39.751 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:05:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:05:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:05:39.753 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:05:49.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:05:49.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:05:49.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:05:49.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:05:49.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:05:49.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:05:49.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:05:49.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:05:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:05:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:05:49.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:06:09.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:06:09.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:06:09.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:06:09.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:06:09.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:06:09.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:06:09.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:06:09.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:06:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:06:09.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:06:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:06:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:06:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:06:39.720 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:06:39.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:06:39.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:06:39.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:06:39.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:06:39.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:06:39.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:06:39.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:06:39.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:06:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:06:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:06:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:06:49.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:06:49.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:06:49.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:06:49.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:06:49.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:06:49.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:06:49.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:06:49.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:06:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:06:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:06:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:07:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:07:09.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:07:09.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:07:09.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:07:09.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:07:09.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:07:09.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:07:09.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:07:09.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:07:09.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:07:09.841 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:07:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:07:39.713 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:07:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:07:39.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:07:39.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:07:39.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:07:39.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:07:39.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:07:39.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:07:39.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:07:39.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:07:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:07:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:07:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:07:49.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:07:49.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:07:49.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:07:49.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:07:49.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:07:49.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:07:49.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:07:49.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:07:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:07:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:07:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:08:09.800 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:08:09.834 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:08:09.834 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:08:09.834 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:08:09.834 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:08:09.834 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:08:09.834 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:08:09.834 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:08:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:08:09.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:08:09.836 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:08:39.708 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:08:39.709 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:08:39.718 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:08:39.718 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:08:39.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:08:39.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:08:39.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:08:39.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:08:39.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:08:39.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:08:39.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:08:39.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:08:39.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.053 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:08:39.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:08:49.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:08:49.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:08:49.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:08:49.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:08:49.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:08:49.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:08:49.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:08:49.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:08:49.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:08:49.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.094 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:08:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:09:09.814 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:09:09.848 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:09:09.848 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:09:09.848 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:09:09.848 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:09:09.848 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:09:09.848 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:09:09.848 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:09:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:09:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:09:09.850 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:09:39.712 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:09:39.712 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:09:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:09:39.720 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:09:39.757 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:09:39.757 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:09:39.757 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:09:39.757 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:09:39.757 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:09:39.757 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:09:39.757 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:09:39.759 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:09:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:09:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:09:49.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:09:49.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:09:49.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:09:49.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:09:49.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:09:49.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:09:49.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:09:49.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:09:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:09:49.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:09:49.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:10:09.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:10:09.829 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:10:09.829 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:10:09.829 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:10:09.829 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:10:09.829 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:10:09.829 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:10:09.829 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:10:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:10:09.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:10:09.831 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:10:39.713 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:10:39.714 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:10:39.721 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:10:39.722 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:10:39.758 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:10:39.758 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:10:39.758 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:10:39.758 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:10:39.758 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:10:39.758 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:10:39.758 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:10:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:10:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:10:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:10:49.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:10:49.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:10:49.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:10:49.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:10:49.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:10:49.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:10:49.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:10:49.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:10:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:10:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:10:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:11:09.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:11:09.833 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:11:09.833 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:11:09.833 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:11:09.833 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:11:09.833 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:11:09.833 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:11:09.833 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:11:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:11:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:11:09.836 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:11:39.717 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:11:39.718 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:11:39.726 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:11:39.726 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:11:39.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:11:39.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:11:39.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:11:39.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:11:39.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:11:39.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:11:39.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:11:39.760 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:11:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:11:39.761 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:11:49.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:11:49.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:11:49.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:11:49.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:11:49.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:11:49.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:11:49.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:11:49.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:11:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:11:49.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:11:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:12:09.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:12:09.832 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:12:09.832 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:12:09.832 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:12:09.832 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:12:09.832 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:12:09.832 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:12:09.832 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:12:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:12:09.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:12:09.834 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:12:39.716 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:12:39.717 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:12:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:12:39.725 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:12:39.759 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:12:39.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:12:39.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:12:39.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:12:39.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:12:39.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:12:39.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:12:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:12:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:12:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:12:49.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:12:49.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:12:49.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:12:49.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:12:49.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:12:49.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:12:49.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:12:49.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:12:49.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:12:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:12:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:13:09.802 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:13:09.835 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:13:09.835 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:13:09.835 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:13:09.835 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:13:09.835 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:13:09.835 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:13:09.835 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:13:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:13:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:13:09.838 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:13:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:13:39.720 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:13:39.729 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:13:39.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:13:39.760 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:13:39.760 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:13:39.760 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:13:39.760 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:13:39.760 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:13:39.760 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:13:39.760 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:13:39.762 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:13:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:13:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:13:49.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:13:49.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:13:49.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:13:49.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:13:49.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:13:49.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:13:49.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:13:49.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:13:49.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:13:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:13:49.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:14:09.803 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:14:09.843 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:14:09.843 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:14:09.843 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:14:09.843 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:14:09.843 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:14:09.843 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:14:09.843 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:14:09.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:14:09.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:14:09.848 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:14:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:14:39.720 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:14:39.728 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:14:39.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:14:39.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:14:39.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:14:39.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:14:39.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:14:39.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:14:39.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:14:39.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:14:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:14:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:14:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:14:49.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:14:49.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:14:49.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:14:49.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:14:49.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:14:49.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:14:49.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:14:49.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:14:49.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:14:49.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:14:49.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:15:09.807 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:15:09.839 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:15:09.839 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:15:09.839 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:15:09.839 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:15:09.839 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:15:09.839 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:15:09.839 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:15:09.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:15:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:15:09.841 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:15:39.722 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:15:39.723 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:15:39.729 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:15:39.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:15:39.761 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:15:39.761 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:15:39.761 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:15:39.761 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:15:39.761 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:15:39.761 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:15:39.761 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:15:39.763 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:15:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:15:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:15:49.764 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:15:49.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:15:49.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:15:49.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:15:49.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:15:49.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:15:49.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:15:49.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:15:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:15:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:15:49.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:16:09.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:16:09.833 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:16:09.833 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:16:09.833 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:16:09.833 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:16:09.833 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:16:09.833 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:16:09.833 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:16:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:16:09.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:16:09.835 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:16:39.727 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:16:39.728 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:16:39.734 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:16:39.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:16:39.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:16:39.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:16:39.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:16:39.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:16:39.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:16:39.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:16:39.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:16:39.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:16:39.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:16:39.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:16:49.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:16:49.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:16:49.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:16:49.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:16:49.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:16:49.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:16:49.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:16:49.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:16:49.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:16:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:16:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:17:09.809 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:17:09.842 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:17:09.842 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:17:09.842 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:17:09.842 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:17:09.842 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:17:09.842 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:17:09.842 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:17:09.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:17:09.845 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:17:09.845 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:17:39.727 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:17:39.728 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:17:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:17:39.735 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:17:39.767 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:17:39.767 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:17:39.767 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:17:39.767 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:17:39.767 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:17:39.767 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:17:39.767 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:17:39.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:17:39.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:17:39.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:17:49.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:17:49.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:17:49.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:17:49.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:17:49.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:17:49.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:17:49.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:17:49.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:17:49.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:17:49.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:17:49.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:18:09.808 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:18:09.839 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:18:09.839 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:18:09.839 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:18:09.839 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:18:09.839 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:18:09.839 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:18:09.839 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:18:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:18:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:18:09.842 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:18:39.723 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:18:39.724 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:18:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:18:39.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:18:39.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:18:39.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:18:39.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:18:39.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:18:39.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:18:39.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:18:39.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:18:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:18:39.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:18:39.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:18:49.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:18:49.801 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:18:49.801 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:18:49.801 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:18:49.801 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:18:49.801 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:18:49.801 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:18:49.801 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:18:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:18:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:18:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:19:09.805 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:19:09.841 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:19:09.841 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:19:09.841 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:19:09.841 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:19:09.841 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:19:09.841 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:19:09.841 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:19:09.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:19:09.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:19:09.844 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:19:39.725 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:19:39.726 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:19:39.737 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:19:39.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:19:39.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:19:39.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:19:39.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:19:39.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:19:39.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:19:39.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:19:39.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:19:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:19:39.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.054 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:19:39.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:19:49.793 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:19:49.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:19:49.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:19:49.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:19:49.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:19:49.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:19:49.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:19:49.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:19:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:19:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.094 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:19:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:20:09.833 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:20:09.867 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:20:09.867 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:20:09.867 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:20:09.867 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:20:09.867 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:20:09.867 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:20:09.867 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:20:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:20:09.870 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.132 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:20:09.870 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:20:39.722 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:20:39.722 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:20:39.729 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:20:39.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:20:39.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:20:39.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:20:39.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:20:39.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:20:39.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:20:39.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:20:39.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:20:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:20:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:20:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:20:49.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:20:49.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:20:49.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:20:49.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:20:49.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:20:49.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:20:49.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:20:49.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:20:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:20:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:20:49.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:21:09.803 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:21:09.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:21:09.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:21:09.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:21:09.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:21:09.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:21:09.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:21:09.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:21:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:21:09.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:21:09.840 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:21:39.722 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:21:39.722 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:21:39.729 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:21:39.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:21:39.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:21:39.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:21:39.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:21:39.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:21:39.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:21:39.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:21:39.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:21:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:21:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:21:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:21:49.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:21:49.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:21:49.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:21:49.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:21:49.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:21:49.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:21:49.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:21:49.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:21:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:21:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:21:49.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:22:09.809 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:22:09.842 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:22:09.842 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:22:09.842 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:22:09.842 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:22:09.842 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:22:09.842 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:22:09.842 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:22:09.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:22:09.845 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:22:09.845 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:22:39.726 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:22:39.727 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:22:39.738 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:22:39.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:22:39.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:22:39.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:22:39.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:22:39.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:22:39.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:22:39.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:22:39.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:22:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:22:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:22:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:22:49.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:22:49.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:22:49.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:22:49.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:22:49.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:22:49.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:22:49.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:22:49.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:22:49.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:22:49.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:22:49.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:23:09.821 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:23:09.854 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:23:09.854 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:23:09.854 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:23:09.854 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:23:09.854 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:23:09.854 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:23:09.854 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:23:09.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:23:09.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:23:09.857 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:23:39.729 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:23:39.729 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:23:39.737 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:23:39.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:23:39.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:23:39.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:23:39.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:23:39.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:23:39.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:23:39.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:23:39.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:23:39.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:23:39.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:23:39.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:23:49.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:23:49.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:23:49.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:23:49.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:23:49.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:23:49.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:23:49.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:23:49.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:23:49.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:23:49.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:23:49.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:24:09.811 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:24:09.847 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:24:09.847 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:24:09.847 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:24:09.847 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:24:09.847 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:24:09.847 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:24:09.847 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:24:09.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:24:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:24:09.850 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:24:39.731 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:24:39.732 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:24:39.740 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:24:39.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:24:39.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:24:39.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:24:39.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:24:39.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:24:39.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:24:39.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:24:39.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:24:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:24:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:24:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:24:49.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:24:49.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:24:49.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:24:49.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:24:49.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:24:49.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:24:49.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:24:49.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:24:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:24:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:24:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:25:09.822 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:25:09.863 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:25:09.863 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:25:09.863 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:25:09.863 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:25:09.863 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:25:09.863 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:25:09.863 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:25:09.867 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:25:09.867 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:25:09.867 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:25:39.729 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:25:39.729 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:25:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:25:39.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:25:39.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:25:39.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:25:39.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:25:39.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:25:39.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:25:39.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:25:39.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:25:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:25:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:25:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:25:49.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:25:49.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:25:49.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:25:49.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:25:49.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:25:49.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:25:49.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:25:49.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:25:49.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:25:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:25:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:26:09.815 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:26:09.847 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:26:09.847 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:26:09.847 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:26:09.847 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:26:09.847 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:26:09.847 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:26:09.847 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:26:09.848 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:26:09.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:26:09.849 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:26:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:26:39.732 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:26:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:26:39.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:26:39.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:26:39.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:26:39.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:26:39.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:26:39.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:26:39.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:26:39.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:26:39.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:26:39.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:26:39.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:26:49.775 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:26:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:26:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:26:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:26:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:26:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:26:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:26:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:26:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:26:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:26:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:27:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:27:09.850 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:27:09.850 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:27:09.850 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:27:09.850 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:27:09.850 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:27:09.850 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:27:09.850 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:27:09.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:27:09.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:27:09.853 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:27:39.734 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:27:39.735 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:27:39.743 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:27:39.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:27:39.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:27:39.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:27:39.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:27:39.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:27:39.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:27:39.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:27:39.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:27:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:27:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:27:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:27:49.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:27:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:27:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:27:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:27:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:27:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:27:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:27:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:27:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:27:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:27:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:28:09.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:28:09.852 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:28:09.852 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:28:09.852 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:28:09.852 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:28:09.852 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:28:09.852 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:28:09.852 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:28:09.854 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:28:09.855 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:28:09.855 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:28:39.736 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:28:39.737 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:28:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:28:39.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:28:39.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:28:39.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:28:39.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:28:39.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:28:39.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:28:39.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:28:39.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:28:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:28:39.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:28:39.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:28:49.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:28:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:28:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:28:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:28:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:28:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:28:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:28:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:28:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:28:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:28:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:29:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:29:09.855 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:29:09.855 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:29:09.855 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:29:09.855 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:29:09.855 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:29:09.855 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:29:09.855 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:29:09.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:29:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:29:09.859 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:29:39.740 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:29:39.740 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:29:39.748 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:29:39.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:29:39.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:29:39.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:29:39.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:29:39.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:29:39.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:29:39.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:29:39.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:29:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:29:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:29:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:29:49.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:29:49.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:29:49.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:29:49.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:29:49.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:29:49.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:29:49.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:29:49.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:29:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:29:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:29:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:30:09.826 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:30:09.866 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:30:09.866 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:30:09.866 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:30:09.866 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:30:09.866 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:30:09.866 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:30:09.866 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:30:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:30:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:30:09.870 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:30:39.742 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:30:39.742 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:30:39.748 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:30:39.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:30:39.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:30:39.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:30:39.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:30:39.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:30:39.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:30:39.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:30:39.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:30:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:30:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:30:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:30:49.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:30:49.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:30:49.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:30:49.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:30:49.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:30:49.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:30:49.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:30:49.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:30:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:30:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:30:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:31:09.823 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:31:09.857 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:31:09.857 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:31:09.857 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:31:09.857 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:31:09.857 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:31:09.857 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:31:09.857 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:31:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:31:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:31:09.859 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:31:39.740 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:31:39.741 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:31:39.748 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:31:39.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:31:39.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:31:39.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:31:39.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:31:39.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:31:39.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:31:39.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:31:39.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:31:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:31:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:31:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:31:49.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:31:49.818 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:31:49.818 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:31:49.818 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:31:49.818 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:31:49.818 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:31:49.818 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:31:49.818 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:31:49.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:31:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:31:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:32:09.821 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:32:09.858 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:32:09.858 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:32:09.858 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:32:09.858 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:32:09.858 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:32:09.858 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:32:09.858 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:32:09.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:32:09.861 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:32:09.861 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:32:39.743 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:32:39.743 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:32:39.750 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:32:39.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:32:39.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:32:39.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:32:39.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:32:39.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:32:39.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:32:39.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:32:39.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:32:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:32:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:32:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:32:49.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:32:49.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:32:49.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:32:49.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:32:49.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:32:49.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:32:49.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:32:49.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:32:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:32:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:32:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:33:09.823 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:33:09.856 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:33:09.856 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:33:09.856 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:33:09.856 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:33:09.856 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:33:09.856 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:33:09.856 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:33:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:33:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:33:09.860 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:33:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:33:39.742 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:34:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:34:39.740 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:34:39.748 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:34:39.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:34:39.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:34:39.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:34:39.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:34:39.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:34:39.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:34:39.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:34:39.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:34:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:34:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:34:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:34:49.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:34:49.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:34:49.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:34:49.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:34:49.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:34:49.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:34:49.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:34:49.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:34:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:34:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:34:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:35:09.826 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:35:09.857 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:35:09.857 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:35:09.857 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:35:09.857 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:35:09.857 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:35:09.857 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:35:09.857 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:35:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:35:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:35:09.860 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:35:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:35:39.742 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:35:39.749 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:35:39.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:35:39.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:35:39.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:35:39.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:35:39.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:35:39.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:35:39.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:35:39.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:35:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:35:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:35:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:35:49.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:35:49.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:35:49.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:35:49.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:35:49.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:35:49.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:35:49.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:35:49.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:35:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:35:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:35:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:36:09.828 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:36:09.865 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:36:09.865 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:36:09.865 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:36:09.865 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:36:09.865 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:36:09.865 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:36:09.865 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:36:09.867 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:36:09.867 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:36:09.868 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:36:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:36:39.740 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:36:39.746 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:36:39.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:36:39.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:36:39.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:36:39.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:36:39.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:36:39.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:36:39.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:36:39.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:36:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:36:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:36:39.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:36:49.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:36:49.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:36:49.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:36:49.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:36:49.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:36:49.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:36:49.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:36:49.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:36:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:36:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:36:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:37:09.822 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:37:09.856 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:37:09.856 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:37:09.856 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:37:09.856 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:37:09.856 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:37:09.856 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:37:09.856 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:37:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:37:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:37:09.859 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:37:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:37:39.741 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:37:39.750 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:37:39.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:37:39.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:37:39.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:37:39.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:37:39.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:37:39.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:37:39.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:37:39.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:37:39.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:37:39.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:37:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:37:49.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:37:49.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:37:49.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:37:49.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:37:49.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:37:49.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:37:49.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:37:49.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:37:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:37:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:37:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:38:09.830 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:38:09.863 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:38:09.863 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:38:09.863 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:38:09.863 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:38:09.863 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:38:09.863 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:38:09.863 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:38:09.866 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:38:09.866 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:38:09.866 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:38:39.737 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:38:39.738 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:38:39.745 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:38:39.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:38:39.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:38:39.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:38:39.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:38:39.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:38:39.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:38:39.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:38:39.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:38:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:38:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:38:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:38:49.781 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:38:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:38:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:38:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:38:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:38:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:38:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:38:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:38:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:38:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:38:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:39:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:39:09.851 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:39:09.851 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:39:09.851 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:39:09.851 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:39:09.851 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:39:09.851 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:39:09.851 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:39:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:39:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:39:09.853 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:39:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:39:39.735 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:39:39.745 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:39:39.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:39:39.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:39:39.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:39:39.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:39:39.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:39:39.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:39:39.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:39:39.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:39:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:39:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:39:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:39:49.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:39:49.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:39:49.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:39:49.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:39:49.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:39:49.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:39:49.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:39:49.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:39:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:39:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:39:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:40:09.825 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:40:09.857 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:40:09.857 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:40:09.857 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:40:09.857 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:40:09.857 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:40:09.857 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:40:09.857 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:40:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:40:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:40:09.859 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:40:39.731 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:40:39.732 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:40:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:40:39.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:40:39.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:40:39.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:40:39.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:40:39.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:40:39.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:40:39.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:40:39.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:40:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:40:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:40:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:40:49.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:40:49.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:40:49.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:40:49.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:40:49.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:40:49.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:40:49.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:40:49.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:40:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:40:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:40:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:41:09.819 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:41:09.850 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:41:09.850 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:41:09.850 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:41:09.850 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:41:09.850 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:41:09.850 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:41:09.850 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:41:09.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:41:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:41:09.853 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:41:39.734 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:41:39.735 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:41:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:41:39.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:41:39.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:41:39.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:41:39.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:41:39.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:41:39.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:41:39.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:41:39.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:41:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:41:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:41:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:41:49.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:41:49.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:41:49.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:41:49.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:41:49.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:41:49.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:41:49.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:41:49.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:41:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:41:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:41:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:42:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:42:09.850 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:42:09.850 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:42:09.850 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:42:09.850 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:42:09.850 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:42:09.850 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:42:09.850 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:42:09.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:42:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:42:09.853 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:42:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:42:39.736 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:42:39.743 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:42:39.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:42:39.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:42:39.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:42:39.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:42:39.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:42:39.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:42:39.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:42:39.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:42:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:42:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:42:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:42:49.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:42:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:42:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:42:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:42:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:42:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:42:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:42:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:42:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:42:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:42:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:43:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:43:09.851 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:43:09.851 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:43:09.851 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:43:09.851 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:43:09.851 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:43:09.851 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:43:09.851 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:43:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:43:09.854 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:43:09.854 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:43:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:43:39.736 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:43:39.743 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:43:39.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:43:39.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:43:39.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:43:39.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:43:39.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:43:39.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:43:39.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:43:39.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:43:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:43:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:43:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:43:49.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:43:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:43:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:43:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:43:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:43:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:43:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:43:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:43:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:43:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:43:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:44:09.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:44:09.854 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:44:09.854 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:44:09.854 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:44:09.854 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:44:09.854 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:44:09.854 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:44:09.854 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:44:09.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:44:09.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:44:09.857 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:44:39.738 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:44:39.739 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:44:39.746 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:44:39.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:44:39.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:44:39.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:44:39.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:44:39.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:44:39.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:44:39.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:44:39.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:44:39.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:44:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:44:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:44:49.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:44:49.818 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:44:49.818 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:44:49.818 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:44:49.818 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:44:49.818 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:44:49.818 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:44:49.818 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:44:49.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:44:49.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:44:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:45:09.821 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:45:09.855 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:45:09.855 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:45:09.855 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:45:09.855 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:45:09.855 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:45:09.855 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:45:09.855 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:45:09.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:45:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:45:09.858 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:45:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:45:39.740 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:45:39.752 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:45:39.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:45:39.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:45:39.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:45:39.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:45:39.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:45:39.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:45:39.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:45:39.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:45:39.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:45:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:45:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:45:49.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:45:49.833 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:45:49.833 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:45:49.833 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:45:49.833 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:45:49.833 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:45:49.833 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:45:49.833 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:45:49.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:45:49.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:45:49.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:46:09.836 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:46:09.875 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:46:09.875 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:46:09.875 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:46:09.875 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:46:09.875 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:46:09.875 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:46:09.875 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:46:09.877 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:46:09.878 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:46:09.878 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:46:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:46:39.740 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:46:39.749 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:46:39.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:46:39.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:46:39.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:46:39.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:46:39.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:46:39.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:46:39.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:46:39.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:46:39.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:46:39.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.051 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:46:39.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:46:49.802 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:46:49.838 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:46:49.838 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:46:49.838 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:46:49.838 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:46:49.838 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:46:49.838 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:46:49.838 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:46:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:46:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.09 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:46:49.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:47:09.841 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:47:09.883 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:47:09.883 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:47:09.883 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:47:09.883 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:47:09.883 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:47:09.883 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:47:09.883 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:47:09.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:47:09.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.137 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:47:09.887 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:47:39.738 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:47:39.739 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:47:39.745 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:47:39.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:47:39.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:47:39.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:47:39.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:47:39.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:47:39.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:47:39.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:47:39.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:47:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:47:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:47:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:47:49.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:47:49.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:47:49.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:47:49.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:47:49.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:47:49.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:47:49.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:47:49.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:47:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:47:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:47:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:48:09.824 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:48:09.860 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:48:09.860 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:48:09.860 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:48:09.860 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:48:09.860 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:48:09.860 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:48:09.860 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:48:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:48:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:48:09.863 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:48:39.734 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:48:39.735 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:48:39.742 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:48:39.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:48:39.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:48:39.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:48:39.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:48:39.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:48:39.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:48:39.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:48:39.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:48:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:48:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:48:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:48:49.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:48:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:48:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:48:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:48:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:48:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:48:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:48:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:48:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:48:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:48:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:49:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:49:09.851 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:49:09.851 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:49:09.851 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:49:09.851 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:49:09.851 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:49:09.851 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:49:09.851 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:49:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:49:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:49:09.853 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:49:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:49:39.736 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:49:39.742 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:49:39.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:49:39.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:49:39.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:49:39.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:49:39.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:49:39.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:49:39.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:49:39.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:49:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:49:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:49:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:49:49.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:49:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:49:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:49:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:49:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:49:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:49:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:49:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:49:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:49:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:49:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:50:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:50:09.849 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:50:09.849 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:50:09.849 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:50:09.849 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:50:09.849 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:50:09.849 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:50:09.849 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:50:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:50:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:50:09.851 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:50:39.732 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:50:39.733 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:50:39.740 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:50:39.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:50:39.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:50:39.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:50:39.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:50:39.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:50:39.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:50:39.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:50:39.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:50:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:50:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:50:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:50:49.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:50:49.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:50:49.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:50:49.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:50:49.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:50:49.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:50:49.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:50:49.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:50:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:50:49.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:50:49.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:51:09.815 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:51:09.847 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:51:09.847 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:51:09.847 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:51:09.847 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:51:09.847 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:51:09.847 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:51:09.847 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:51:09.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:51:09.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:51:09.850 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:51:39.731 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:51:39.731 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:51:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:51:39.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:51:39.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:51:39.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:51:39.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:51:39.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:51:39.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:51:39.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:51:39.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:51:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:51:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:51:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:51:49.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:51:49.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:51:49.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:51:49.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:51:49.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:51:49.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:51:49.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:51:49.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:51:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:51:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:51:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:52:09.819 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:52:09.851 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:52:09.851 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:52:09.851 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:52:09.851 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:52:09.851 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:52:09.851 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:52:09.851 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:52:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:52:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:52:09.854 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:52:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:52:39.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:52:39.768 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:52:39.768 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:52:39.768 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:52:39.768 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:52:39.768 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:52:39.768 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:52:39.768 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:52:39.770 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:52:39.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:52:39.771 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:52:49.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:52:49.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:52:49.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:52:49.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:52:49.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:52:49.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:52:49.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:52:49.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:52:49.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:52:49.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:52:49.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:53:09.813 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:53:09.849 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:53:09.849 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:53:09.849 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:53:09.849 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:53:09.849 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:53:09.849 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:53:09.849 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:53:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:53:09.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:53:09.852 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:53:39.734 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:53:39.734 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:53:39.742 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:53:39.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:53:39.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:53:39.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:53:39.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:53:39.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:53:39.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:53:39.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:53:39.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:53:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:53:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:53:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:53:49.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:53:49.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:53:49.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:53:49.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:53:49.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:53:49.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:53:49.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:53:49.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:53:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:53:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:53:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:54:09.824 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:54:09.861 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:54:09.861 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:54:09.861 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:54:09.861 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:54:09.861 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:54:09.861 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:54:09.861 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:54:09.863 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:54:09.864 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:54:09.864 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:54:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:54:39.736 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:54:39.743 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:54:39.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:54:39.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:54:39.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:54:39.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:54:39.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:54:39.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:54:39.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:54:39.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:54:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:54:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:54:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:54:49.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:54:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:54:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:54:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:54:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:54:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:54:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:54:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:54:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:54:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:54:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:55:09.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:55:09.854 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:55:09.854 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:55:09.854 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:55:09.854 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:55:09.854 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:55:09.854 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:55:09.854 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:55:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:55:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:55:09.859 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:55:39.731 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:55:39.732 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:55:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:55:39.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:55:39.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:55:39.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:55:39.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:55:39.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:55:39.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:55:39.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:55:39.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:55:39.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:55:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:55:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:55:49.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:55:49.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:55:49.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:55:49.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:55:49.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:55:49.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:55:49.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:55:49.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:55:49.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:55:49.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:55:49.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:56:09.811 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:56:09.846 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:56:09.846 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:56:09.846 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:56:09.846 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:56:09.846 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:56:09.846 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:56:09.846 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:56:09.848 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:56:09.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:56:09.849 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:56:39.731 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:56:39.732 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:56:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:56:39.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:56:39.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:56:39.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:56:39.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:56:39.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:56:39.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:56:39.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:56:39.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:56:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:56:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:56:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:56:49.779 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:56:49.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:56:49.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:56:49.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:56:49.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:56:49.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:56:49.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:56:49.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:56:49.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:56:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:56:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:57:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:57:09.849 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:57:09.849 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:57:09.849 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:57:09.849 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:57:09.849 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:57:09.849 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:57:09.849 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:57:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:57:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:57:09.852 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:57:39.733 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:57:39.734 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:57:39.740 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:57:39.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:57:39.774 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:57:39.774 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:57:39.774 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:57:39.774 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:57:39.774 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:57:39.774 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:57:39.774 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:57:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:57:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:57:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:57:49.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:57:49.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:57:49.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:57:49.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:57:49.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:57:49.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:57:49.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:57:49.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:57:49.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:57:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:57:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:58:09.815 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:58:09.848 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:58:09.848 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:58:09.848 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:58:09.848 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:58:09.848 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:58:09.848 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:58:09.848 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:58:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:58:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:58:09.851 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:58:39.732 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:58:39.733 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:58:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:58:39.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:58:39.776 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:58:39.776 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:58:39.776 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:58:39.776 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:58:39.776 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:58:39.776 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:58:39.776 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:58:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:58:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:58:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:58:49.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:58:49.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:58:49.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:58:49.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:58:49.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:58:49.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:58:49.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:58:49.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:58:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:58:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:58:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:59:09.819 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:59:09.855 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:59:09.855 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:59:09.855 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:59:09.855 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:59:09.855 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:59:09.855 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:59:09.855 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:59:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:59:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 05:59:09.858 139328 ERROR oslo_service.periodic_task 
>2018-06-25 05:59:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:59:39.730 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 05:59:39.738 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 05:59:39.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:59:39.770 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:59:39.770 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:59:39.770 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:59:39.770 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:59:39.770 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:59:39.770 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:59:39.770 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:59:39.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:59:39.772 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:59:39.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 05:59:49.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 05:59:49.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 05:59:49.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 05:59:49.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 05:59:49.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 05:59:49.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 05:59:49.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 05:59:49.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 05:59:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 05:59:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 05:59:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:00:09.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:00:09.853 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:00:09.853 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:00:09.853 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:00:09.853 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:00:09.853 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:00:09.853 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:00:09.853 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:00:09.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:00:09.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:00:09.856 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:00:39.728 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:00:39.729 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:00:39.737 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:00:39.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:00:39.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:00:39.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:00:39.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:00:39.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:00:39.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:00:39.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:00:39.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:00:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:00:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:00:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:00:49.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:00:49.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:00:49.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:00:49.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:00:49.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:00:49.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:00:49.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:00:49.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:00:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:00:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:00:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:01:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:01:09.850 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:01:09.850 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:01:09.850 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:01:09.850 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:01:09.850 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:01:09.850 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:01:09.850 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:01:09.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:01:09.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:01:09.852 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:01:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:01:39.725 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:01:39.732 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:01:39.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:01:39.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:01:39.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:01:39.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:01:39.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:01:39.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:01:39.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:01:39.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:01:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:01:39.779 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.046 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:01:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:01:49.780 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:01:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:01:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:01:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:01:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:01:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:01:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:01:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:01:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:01:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:01:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:02:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:02:09.850 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:02:09.850 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:02:09.850 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:02:09.850 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:02:09.850 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:02:09.850 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:02:09.850 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:02:09.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:02:09.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:02:09.852 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:02:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:02:39.725 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:02:39.742 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:02:39.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:02:39.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:02:39.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:02:39.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:02:39.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:02:39.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:02:39.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:02:39.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:02:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:02:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.053 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:02:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:02:49.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:02:49.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:02:49.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:02:49.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:02:49.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:02:49.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:02:49.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:02:49.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:02:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:02:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.091 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:02:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:03:09.835 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:03:09.866 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:03:09.866 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:03:09.866 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:03:09.866 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:03:09.866 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:03:09.866 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:03:09.866 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:03:09.868 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:03:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:03:09.869 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:03:39.721 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:03:39.721 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:03:39.728 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:03:39.729 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:03:39.763 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:03:39.763 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:03:39.763 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:03:39.763 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:03:39.763 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:03:39.763 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:03:39.763 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:03:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:03:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:03:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:03:49.766 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:03:49.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:03:49.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:03:49.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:03:49.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:03:49.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:03:49.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:03:49.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:03:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:03:49.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:03:49.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:04:09.803 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:04:09.835 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:04:09.835 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:04:09.835 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:04:09.835 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:04:09.835 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:04:09.835 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:04:09.835 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:04:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:04:09.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:04:09.837 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:04:39.719 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:04:39.719 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:04:39.727 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:04:39.728 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:04:39.765 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:04:39.765 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:04:39.765 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:04:39.765 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:04:39.765 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:04:39.765 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:04:39.765 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:04:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:04:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:04:39.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:04:49.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:04:49.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:04:49.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:04:49.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:04:49.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:04:49.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:04:49.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:04:49.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:04:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:04:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:04:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:05:09.805 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:05:09.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:05:09.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:05:09.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:05:09.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:05:09.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:05:09.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:05:09.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:05:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:05:09.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:05:09.840 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:05:39.721 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:05:39.722 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:05:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:05:39.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:05:39.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:05:39.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:05:39.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:05:39.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:05:39.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:05:39.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:05:39.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:05:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:05:39.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:05:39.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:05:49.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:05:49.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:05:49.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:05:49.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:05:49.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:05:49.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:05:49.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:05:49.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:05:49.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:05:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:05:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:06:09.808 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:06:09.842 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:06:09.842 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:06:09.842 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:06:09.842 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:06:09.842 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:06:09.842 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:06:09.842 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:06:09.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:06:09.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:06:09.844 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:06:39.726 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:06:39.727 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:06:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:06:39.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:06:39.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:06:39.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:06:39.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:06:39.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:06:39.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:06:39.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:06:39.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:06:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:06:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:06:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:06:49.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:06:49.813 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:06:49.813 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:06:49.813 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:06:49.813 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:06:49.813 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:06:49.813 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:06:49.813 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:06:49.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:06:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:06:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:07:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:07:09.848 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:07:09.848 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:07:09.848 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:07:09.848 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:07:09.848 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:07:09.848 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:07:09.848 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:07:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:07:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:07:09.850 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:07:39.722 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:07:39.723 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:07:39.729 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:07:39.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:07:39.762 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:07:39.762 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:07:39.762 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:07:39.762 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:07:39.762 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:07:39.762 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:07:39.762 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:07:39.764 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:07:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:07:39.765 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:07:49.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:07:49.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:07:49.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:07:49.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:07:49.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:07:49.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:07:49.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:07:49.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:07:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:07:49.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:07:49.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:08:09.803 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:08:09.841 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:08:09.841 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:08:09.841 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:08:09.841 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:08:09.841 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:08:09.841 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:08:09.841 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:08:09.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:08:09.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:08:09.844 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:08:39.725 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:08:39.726 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:08:39.733 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:08:39.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:08:39.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:08:39.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:08:39.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:08:39.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:08:39.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:08:39.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:08:39.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:08:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:08:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.051 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:08:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:08:49.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:08:49.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:08:49.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:08:49.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:08:49.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:08:49.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:08:49.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:08:49.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:08:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:08:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.087 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:08:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:09:09.822 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:09:09.859 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:09:09.859 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:09:09.859 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:09:09.859 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:09:09.859 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:09:09.859 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:09:09.859 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:09:09.861 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:09:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.128 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:09:09.862 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:09:39.723 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:09:39.724 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:09:39.731 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:09:39.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:09:39.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:09:39.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:09:39.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:09:39.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:09:39.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:09:39.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:09:39.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:09:39.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:09:39.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:09:39.767 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:09:49.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:09:49.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:09:49.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:09:49.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:09:49.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:09:49.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:09:49.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:09:49.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:09:49.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:09:49.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:09:49.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:10:09.806 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:10:09.840 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:10:09.840 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:10:09.840 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:10:09.840 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:10:09.840 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:10:09.840 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:10:09.840 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:10:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:10:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:10:09.843 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:10:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:10:39.724 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:10:39.732 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:10:39.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:10:39.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:10:39.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:10:39.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:10:39.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:10:39.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:10:39.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:10:39.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:10:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:10:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:10:39.776 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:10:49.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:10:49.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:10:49.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:10:49.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:10:49.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:10:49.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:10:49.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:10:49.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:10:49.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:10:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:10:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:11:09.814 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:11:09.849 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:11:09.849 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:11:09.849 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:11:09.849 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:11:09.849 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:11:09.849 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:11:09.849 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:11:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:11:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:11:09.852 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:11:39.723 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:11:39.724 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:11:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:11:39.730 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:11:39.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:11:39.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:11:39.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:11:39.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:11:39.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:11:39.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:11:39.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:11:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:11:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:11:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:11:49.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:11:49.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:11:49.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:11:49.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:11:49.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:11:49.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:11:49.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:11:49.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:11:49.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:11:49.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:11:49.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:12:09.806 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:12:09.839 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:12:09.839 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:12:09.839 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:12:09.839 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:12:09.839 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:12:09.839 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:12:09.839 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:12:09.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:12:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:12:09.842 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:12:39.723 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:12:39.724 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:12:39.731 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:12:39.731 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:12:39.764 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:12:39.764 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:12:39.764 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:12:39.764 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:12:39.764 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:12:39.764 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:12:39.764 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:12:39.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:12:39.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:12:39.766 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:12:49.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:12:49.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:12:49.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:12:49.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:12:49.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:12:49.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:12:49.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:12:49.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:12:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:12:49.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:12:49.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:13:09.806 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:13:09.840 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:13:09.840 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:13:09.840 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:13:09.840 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:13:09.840 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:13:09.840 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:13:09.840 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:13:09.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:13:09.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:13:09.843 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:13:39.724 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:13:39.725 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:13:39.731 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:13:39.732 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:13:39.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:13:39.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:13:39.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:13:39.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:13:39.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:13:39.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:13:39.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:13:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:13:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:13:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:13:49.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:13:49.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:13:49.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:13:49.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:13:49.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:13:49.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:13:49.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:13:49.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:13:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:13:49.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:13:49.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:14:09.805 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:14:09.840 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:14:09.840 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:14:09.840 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:14:09.840 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:14:09.840 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:14:09.840 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:14:09.840 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:14:09.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:14:09.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:14:09.844 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:14:39.726 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:14:39.726 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:14:39.733 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:14:39.734 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:14:39.766 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:14:39.766 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:14:39.766 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:14:39.766 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:14:39.766 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:14:39.766 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:14:39.766 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:14:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:14:39.768 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:14:39.769 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:14:49.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:14:49.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:14:49.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:14:49.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:14:49.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:14:49.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:14:49.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:14:49.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:14:49.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:14:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:14:49.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:15:09.809 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:15:09.843 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:15:09.843 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:15:09.843 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:15:09.843 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:15:09.843 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:15:09.843 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:15:09.843 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:15:09.845 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:15:09.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:15:09.846 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:15:39.727 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:15:39.728 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:15:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:15:39.741 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:15:39.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:15:39.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:15:39.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:15:39.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:15:39.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:15:39.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:15:39.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:15:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:15:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:15:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:15:49.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:15:49.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:15:49.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:15:49.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:15:49.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:15:49.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:15:49.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:15:49.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:15:49.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:15:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:15:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:16:09.814 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:16:09.848 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:16:09.848 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:16:09.848 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:16:09.848 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:16:09.848 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:16:09.848 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:16:09.848 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:16:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:16:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:16:09.851 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:16:39.732 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:16:39.733 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:16:39.740 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:16:39.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:16:39.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:16:39.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:16:39.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:16:39.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:16:39.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:16:39.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:16:39.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:16:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:16:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:16:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:16:49.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:16:49.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:16:49.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:16:49.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:16:49.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:16:49.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:16:49.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:16:49.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:16:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:16:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:16:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:17:09.823 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:17:09.858 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:17:09.858 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:17:09.858 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:17:09.858 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:17:09.858 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:17:09.858 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:17:09.858 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:17:09.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:17:09.861 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:17:09.861 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:17:39.732 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:17:39.733 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:17:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:17:39.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:17:39.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:17:39.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:17:39.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:17:39.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:17:39.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:17:39.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:17:39.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:17:39.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:17:39.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:17:39.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:17:49.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:17:49.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:17:49.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:17:49.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:17:49.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:17:49.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:17:49.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:17:49.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:17:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:17:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:17:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:18:09.823 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:18:09.856 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:18:09.856 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:18:09.856 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:18:09.856 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:18:09.856 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:18:09.856 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:18:09.856 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:18:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:18:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:18:09.859 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:18:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:18:39.731 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:18:39.737 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:18:39.737 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:18:39.773 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:18:39.773 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:18:39.773 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:18:39.773 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:18:39.773 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:18:39.773 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:18:39.773 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:18:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:18:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:18:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:18:49.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:18:49.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:18:49.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:18:49.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:18:49.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:18:49.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:18:49.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:18:49.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:18:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:18:49.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:18:49.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:19:09.820 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:19:09.854 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:19:09.854 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:19:09.854 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:19:09.854 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:19:09.854 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:19:09.854 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:19:09.854 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:19:09.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:19:09.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:19:09.856 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:19:39.728 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:19:39.728 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:19:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:19:39.736 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:19:39.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:19:39.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:19:39.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:19:39.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:19:39.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:19:39.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:19:39.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:19:39.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:19:39.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:19:39.774 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:19:49.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:19:49.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:19:49.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:19:49.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:19:49.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:19:49.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:19:49.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:19:49.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:19:49.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:19:49.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:19:49.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:20:09.812 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:20:09.846 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:20:09.846 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:20:09.846 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:20:09.846 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:20:09.846 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:20:09.846 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:20:09.846 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:20:09.848 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:20:09.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:20:09.849 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:20:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:20:39.731 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:20:39.737 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:20:39.738 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:20:39.771 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:20:39.771 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:20:39.771 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:20:39.771 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:20:39.771 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:20:39.771 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:20:39.771 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:20:39.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:20:39.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:20:39.773 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:20:49.774 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:20:49.808 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:20:49.808 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:20:49.808 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:20:49.808 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:20:49.808 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:20:49.808 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:20:49.808 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:20:49.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:20:49.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:20:49.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:21:09.812 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:21:09.845 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:21:09.845 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:21:09.845 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:21:09.845 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:21:09.845 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:21:09.845 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:21:09.845 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:21:09.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:21:09.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:21:09.848 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:21:39.729 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:21:39.730 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:21:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:21:39.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:21:39.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:21:39.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:21:39.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:21:39.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:21:39.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:21:39.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:21:39.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:21:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:21:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:21:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:21:49.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:21:49.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:21:49.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:21:49.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:21:49.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:21:49.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:21:49.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:21:49.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:21:49.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:21:49.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:21:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:22:09.814 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:22:09.848 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:22:09.848 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:22:09.848 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:22:09.848 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:22:09.848 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:22:09.848 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:22:09.848 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:22:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:22:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:22:09.851 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:22:39.733 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:22:39.734 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:22:39.742 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:22:39.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:22:39.777 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:22:39.777 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:22:39.777 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:22:39.777 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:22:39.777 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:22:39.777 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:22:39.777 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:22:39.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:22:39.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:22:39.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:22:49.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:22:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:22:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:22:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:22:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:22:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:22:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:22:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:22:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:22:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:22:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:23:09.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:23:09.851 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:23:09.851 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:23:09.851 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:23:09.851 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:23:09.851 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:23:09.851 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:23:09.851 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:23:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:23:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:23:09.853 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:23:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:23:39.736 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:23:39.746 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:23:39.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:23:39.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:23:39.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:23:39.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:23:39.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:23:39.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:23:39.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:23:39.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:23:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:23:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:23:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:23:49.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:23:49.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:23:49.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:23:49.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:23:49.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:23:49.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:23:49.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:23:49.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:23:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:23:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:23:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:24:09.828 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:24:09.873 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:24:09.873 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:24:09.873 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:24:09.873 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:24:09.873 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:24:09.873 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:24:09.873 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:24:09.876 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:24:09.876 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:24:09.877 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:24:39.738 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:24:39.739 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:24:39.748 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:24:39.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:24:39.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:24:39.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:24:39.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:24:39.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:24:39.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:24:39.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:24:39.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:24:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:24:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.046 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:24:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:24:49.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:24:49.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:24:49.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:24:49.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:24:49.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:24:49.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:24:49.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:24:49.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:24:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:24:49.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:24:49.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:25:09.831 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:25:09.879 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:25:09.879 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:25:09.879 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:25:09.879 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:25:09.879 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:25:09.879 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:25:09.879 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:25:09.883 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:25:09.883 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.134 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:25:09.884 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:25:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:25:39.736 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:25:39.743 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:25:39.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:25:39.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:25:39.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:25:39.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:25:39.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:25:39.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:25:39.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:25:39.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:25:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:25:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:25:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:25:49.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:25:49.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:25:49.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:25:49.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:25:49.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:25:49.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:25:49.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:25:49.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:25:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:25:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:25:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:26:09.829 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:26:09.865 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:26:09.865 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:26:09.865 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:26:09.865 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:26:09.865 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:26:09.865 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:26:09.865 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:26:09.867 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:26:09.867 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.124 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:26:09.868 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:26:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:26:39.740 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:26:39.747 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:26:39.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:26:39.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:26:39.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:26:39.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:26:39.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:26:39.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:26:39.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:26:39.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:26:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:26:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:26:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:26:49.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:26:49.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:26:49.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:26:49.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:26:49.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:26:49.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:26:49.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:26:49.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:26:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:26:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:26:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:27:09.824 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:27:09.857 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:27:09.857 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:27:09.857 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:27:09.857 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:27:09.857 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:27:09.857 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:27:09.857 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:27:09.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:27:09.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:27:09.860 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:27:39.742 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:27:39.742 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:27:39.749 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:27:39.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:27:39.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:27:39.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:27:39.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:27:39.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:27:39.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:27:39.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:27:39.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:27:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:27:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:27:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:27:49.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:27:49.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:27:49.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:27:49.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:27:49.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:27:49.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:27:49.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:27:49.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:27:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:27:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:27:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:28:09.822 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:28:09.856 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:28:09.856 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:28:09.856 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:28:09.856 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:28:09.856 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:28:09.856 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:28:09.856 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:28:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:28:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:28:09.858 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:28:39.740 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:28:39.740 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:28:39.747 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:28:39.748 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:28:39.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:28:39.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:28:39.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:28:39.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:28:39.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:28:39.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:28:39.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:28:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:28:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:28:39.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:28:49.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:28:49.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:28:49.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:28:49.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:28:49.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:28:49.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:28:49.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:28:49.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:28:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:28:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:28:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:29:09.824 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:29:09.857 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:29:09.857 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:29:09.857 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:29:09.857 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:29:09.857 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:29:09.857 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:29:09.857 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:29:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:29:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:29:09.859 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:29:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:29:39.742 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:29:39.748 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:29:39.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:29:39.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:29:39.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:29:39.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:29:39.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:29:39.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:29:39.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:29:39.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:29:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:29:39.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.049 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:29:39.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:29:49.799 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:29:49.833 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:29:49.833 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:29:49.833 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:29:49.833 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:29:49.833 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:29:49.833 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:29:49.833 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:29:49.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:29:49.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.087 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:29:49.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:30:09.837 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:30:09.877 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:30:09.877 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:30:09.877 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:30:09.877 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:30:09.877 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:30:09.877 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:30:09.877 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:30:09.879 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:30:09.879 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:30:09.880 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:30:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:30:39.743 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:30:39.749 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:30:39.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:30:39.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:30:39.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:30:39.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:30:39.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:30:39.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:30:39.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:30:39.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:30:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:30:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:30:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:30:49.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:30:49.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:30:49.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:30:49.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:30:49.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:30:49.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:30:49.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:30:49.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:30:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:30:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:30:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:31:09.828 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:31:09.859 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:31:09.859 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:31:09.859 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:31:09.859 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:31:09.859 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:31:09.859 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:31:09.859 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:31:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:31:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:31:09.863 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:31:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:31:39.745 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:31:39.752 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:31:39.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:31:39.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:31:39.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:31:39.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:31:39.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:31:39.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:31:39.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:31:39.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:31:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:31:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:31:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:31:49.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:31:49.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:31:49.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:31:49.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:31:49.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:31:49.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:31:49.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:31:49.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:31:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:31:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:31:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:32:09.824 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:32:09.859 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:32:09.859 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:32:09.859 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:32:09.859 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:32:09.859 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:32:09.859 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:32:09.859 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:32:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:32:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:32:09.862 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:32:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:32:39.745 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:32:39.753 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:32:39.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:32:39.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:32:39.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:32:39.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:32:39.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:32:39.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:32:39.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:32:39.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:32:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:32:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:32:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:32:49.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:32:49.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:32:49.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:32:49.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:32:49.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:32:49.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:32:49.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:32:49.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:32:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:32:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:32:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:33:09.826 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:33:09.862 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:33:09.862 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:33:09.862 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:33:09.862 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:33:09.862 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:33:09.862 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:33:09.862 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:33:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:33:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:33:09.865 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:33:39.747 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:33:39.747 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:33:39.754 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:33:39.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:33:39.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:33:39.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:33:39.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:33:39.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:33:39.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:33:39.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:33:39.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:33:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:33:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:33:39.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:33:49.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:33:49.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:33:49.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:33:49.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:33:49.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:33:49.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:33:49.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:33:49.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:33:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:33:49.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:33:49.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:34:09.831 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:34:09.863 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:34:09.863 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:34:09.863 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:34:09.863 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:34:09.863 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:34:09.863 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:34:09.863 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:34:09.864 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:34:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:34:09.865 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:34:39.746 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:34:39.747 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:34:39.753 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:34:39.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:34:39.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:34:39.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:34:39.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:34:39.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:34:39.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:34:39.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:34:39.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:34:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:34:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:34:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:34:49.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:34:49.843 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:34:49.843 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:34:49.843 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:34:49.843 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:34:49.843 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:34:49.843 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:34:49.843 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:34:49.845 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:34:49.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.092 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:34:49.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:35:09.847 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:35:09.881 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:35:09.881 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:35:09.881 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:35:09.881 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:35:09.881 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:35:09.881 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:35:09.881 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:35:09.884 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:35:09.885 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.13 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:35:09.885 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:35:39.747 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:35:39.747 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:35:39.760 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:35:39.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:35:39.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:35:39.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:35:39.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:35:39.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:35:39.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:35:39.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:35:39.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:35:39.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:35:39.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:35:39.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:35:49.803 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:35:49.839 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:35:49.839 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:35:49.839 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:35:49.839 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:35:49.839 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:35:49.839 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:35:49.839 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:35:49.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:35:49.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:35:49.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:36:09.842 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:36:09.877 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:36:09.877 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:36:09.877 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:36:09.877 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:36:09.877 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:36:09.877 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:36:09.877 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:36:09.879 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:36:09.879 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:36:09.880 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:36:39.751 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:36:39.752 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:37:39.749 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:37:39.750 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:37:39.757 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:37:39.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:37:39.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:37:39.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:37:39.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:37:39.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:37:39.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:37:39.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:37:39.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:37:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:37:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:37:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:37:49.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:37:49.849 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:37:49.849 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:37:49.849 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:37:49.849 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:37:49.849 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:37:49.849 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:37:49.849 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:37:49.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:37:49.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.094 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:37:49.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:38:09.852 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:38:09.885 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:38:09.885 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:38:09.885 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:38:09.885 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:38:09.885 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:38:09.885 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:38:09.885 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:38:09.887 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:38:09.887 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:38:09.887 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:38:39.749 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:38:39.750 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:38:39.757 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:38:39.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:38:39.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:38:39.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:38:39.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:38:39.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:38:39.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:38:39.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:38:39.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:38:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:38:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:38:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:38:49.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:38:49.834 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:38:49.834 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:38:49.834 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:38:49.834 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:38:49.834 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:38:49.834 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:38:49.834 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:38:49.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:38:49.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:38:49.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:39:09.838 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:39:09.871 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:39:09.871 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:39:09.871 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:39:09.871 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:39:09.871 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:39:09.871 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:39:09.871 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:39:09.872 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:39:09.872 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:39:09.873 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:39:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:39:39.745 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:39:39.753 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:39:39.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:39:39.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:39:39.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:39:39.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:39:39.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:39:39.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:39:39.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:39:39.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:39:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:39:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:39:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:39:49.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:39:49.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:39:49.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:39:49.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:39:49.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:39:49.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:39:49.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:39:49.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:39:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:39:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:39:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:40:09.829 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:40:09.864 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:40:09.864 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:40:09.864 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:40:09.864 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:40:09.864 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:40:09.864 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:40:09.864 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:40:09.866 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:40:09.866 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:40:09.866 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:40:39.748 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:40:39.748 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:40:39.760 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:40:39.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:40:39.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:40:39.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:40:39.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:40:39.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:40:39.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:40:39.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:40:39.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:40:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:40:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:40:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:40:49.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:40:49.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:40:49.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:40:49.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:40:49.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:40:49.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:40:49.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:40:49.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:40:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:40:49.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:40:49.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:41:09.830 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:41:09.865 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:41:09.865 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:41:09.865 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:41:09.865 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:41:09.865 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:41:09.865 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:41:09.865 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:41:09.867 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:41:09.867 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:41:09.867 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:41:39.749 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:41:39.749 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:41:39.756 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:41:39.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:41:39.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:41:39.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:41:39.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:41:39.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:41:39.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:41:39.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:41:39.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:41:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:41:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:41:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:41:49.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:41:49.832 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:41:49.832 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:41:49.832 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:41:49.832 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:41:49.832 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:41:49.832 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:41:49.832 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:41:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:41:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:41:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:42:09.835 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:42:09.867 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:42:09.867 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:42:09.867 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:42:09.867 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:42:09.867 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:42:09.867 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:42:09.867 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:42:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:42:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:42:09.870 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:42:39.751 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:42:39.752 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:42:39.759 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:42:39.760 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:42:39.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:42:39.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:42:39.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:42:39.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:42:39.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:42:39.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:42:39.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:42:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:42:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:42:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:42:49.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:42:49.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:42:49.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:42:49.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:42:49.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:42:49.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:42:49.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:42:49.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:42:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:42:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:42:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:43:09.833 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:43:09.866 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:43:09.866 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:43:09.866 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:43:09.866 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:43:09.866 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:43:09.866 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:43:09.866 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:43:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:43:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:43:09.869 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:43:39.751 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:43:39.751 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:43:39.757 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:43:39.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:43:39.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:43:39.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:43:39.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:43:39.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:43:39.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:43:39.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:43:39.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:43:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:43:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:43:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:43:49.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:43:49.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:43:49.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:43:49.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:43:49.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:43:49.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:43:49.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:43:49.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:43:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:43:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:43:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:44:09.834 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:44:09.867 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:44:09.867 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:44:09.867 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:44:09.867 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:44:09.867 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:44:09.867 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:44:09.867 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:44:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:44:09.870 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:44:09.870 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:44:39.751 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:44:39.752 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:44:39.758 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:44:39.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:44:39.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:44:39.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:44:39.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:44:39.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:44:39.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:44:39.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:44:39.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:44:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:44:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:44:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:44:49.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:44:49.829 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:44:49.829 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:44:49.829 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:44:49.829 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:44:49.829 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:44:49.829 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:44:49.829 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:44:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:44:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:44:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:45:09.833 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:45:09.871 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:45:09.871 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:45:09.871 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:45:09.871 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:45:09.871 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:45:09.871 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:45:09.871 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:45:09.873 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:45:09.874 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:45:09.874 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:45:39.756 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:45:39.756 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:45:39.770 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:45:39.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:45:39.805 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:45:39.805 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:45:39.805 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:45:39.805 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:45:39.805 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:45:39.805 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:45:39.805 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:45:39.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:45:39.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:45:39.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:45:49.809 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:45:49.845 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:45:49.845 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:45:49.845 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:45:49.845 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:45:49.845 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:45:49.845 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:45:49.845 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:45:49.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:45:49.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:45:49.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:46:09.850 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:46:09.884 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:46:09.884 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:46:09.884 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:46:09.884 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:46:09.884 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:46:09.884 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:46:09.884 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:46:09.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:46:09.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:46:09.886 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:46:39.758 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:46:39.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:46:39.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:46:39.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:46:39.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:46:39.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:46:39.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:46:39.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:46:39.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:46:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:46:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:46:39.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:46:49.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:46:49.849 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:46:49.849 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:46:49.849 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:46:49.849 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:46:49.849 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:46:49.849 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:46:49.849 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:46:49.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:46:49.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.093 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:46:49.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:47:09.853 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:47:09.886 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:47:09.886 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:47:09.886 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:47:09.886 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:47:09.886 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:47:09.886 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:47:09.886 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:47:09.888 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:47:09.888 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:47:09.888 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:47:39.760 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:47:39.760 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:47:39.769 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:47:39.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:47:39.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:47:39.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:47:39.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:47:39.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:47:39.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:47:39.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:47:39.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:47:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:47:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:47:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:47:49.806 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:47:49.843 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:47:49.843 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:47:49.843 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:47:49.843 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:47:49.843 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:47:49.843 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:47:49.843 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:47:49.845 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:47:49.845 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:47:49.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:48:09.846 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:48:09.879 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:48:09.879 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:48:09.879 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:48:09.879 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:48:09.879 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:48:09.879 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:48:09.879 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:48:09.881 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:48:09.881 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:48:09.881 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:48:39.763 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:48:39.764 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:48:39.771 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:48:39.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:48:39.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:48:39.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:48:39.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:48:39.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:48:39.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:48:39.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:48:39.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:48:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:48:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:48:39.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:48:49.807 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:48:49.842 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:48:49.842 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:48:49.842 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:48:49.842 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:48:49.842 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:48:49.842 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:48:49.842 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:48:49.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:48:49.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:48:49.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:49:09.847 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:49:09.880 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:49:09.880 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:49:09.880 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:49:09.880 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:49:09.880 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:49:09.880 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:49:09.880 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:49:09.882 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:49:09.882 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:49:09.882 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:49:39.764 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:49:39.764 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:49:39.771 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:49:39.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:49:39.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:49:39.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:49:39.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:49:39.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:49:39.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:49:39.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:49:39.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:49:39.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:49:39.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:49:39.808 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:49:49.808 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:49:49.841 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:49:49.841 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:49:49.841 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:49:49.841 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:49:49.841 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:49:49.841 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:49:49.841 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:49:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:49:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:49:49.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:50:09.844 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:50:09.896 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:50:09.896 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:50:09.896 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:50:09.896 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:50:09.896 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:50:09.896 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:50:09.896 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:50:09.899 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:50:09.899 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.127 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:50:09.900 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:50:39.761 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:50:39.762 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:50:39.771 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:50:39.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:50:39.807 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:50:39.807 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:50:39.807 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:50:39.807 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:50:39.807 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:50:39.807 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:50:39.807 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:50:39.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:50:39.809 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:50:39.810 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:50:49.810 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:50:49.844 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:50:49.844 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:50:49.844 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:50:49.844 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:50:49.844 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:50:49.844 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:50:49.844 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:50:49.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:50:49.847 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:50:49.848 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:51:09.848 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:51:09.880 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:51:09.880 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:51:09.880 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:51:09.880 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:51:09.880 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:51:09.880 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:51:09.880 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:51:09.882 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:51:09.882 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:51:09.883 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:51:39.764 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:51:39.765 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:51:39.773 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:51:39.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:51:39.817 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:51:39.817 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:51:39.817 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:51:39.817 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:51:39.817 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:51:39.817 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:51:39.817 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:51:39.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:51:39.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.046 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:51:39.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:51:49.822 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:51:49.863 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:51:49.863 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:51:49.863 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:51:49.863 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:51:49.863 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:51:49.863 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:51:49.863 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:51:49.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:51:49.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.091 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:51:49.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:52:09.866 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:52:09.902 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:52:09.902 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:52:09.902 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:52:09.902 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:52:09.902 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:52:09.902 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:52:09.902 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:52:09.904 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:52:09.905 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.131 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:52:09.905 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:52:39.766 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:52:39.768 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:52:39.776 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:52:39.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:52:39.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:52:39.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:52:39.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:52:39.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:52:39.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:52:39.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:52:39.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:52:39.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:52:39.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:52:39.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:52:49.816 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:52:49.849 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:52:49.849 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:52:49.849 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:52:49.849 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:52:49.849 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:52:49.849 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:52:49.849 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:52:49.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:52:49.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:52:49.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:53:09.852 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:53:09.884 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:53:09.884 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:53:09.884 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:53:09.884 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:53:09.884 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:53:09.884 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:53:09.884 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:53:09.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:53:09.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:53:09.887 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:53:39.768 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:53:39.769 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:53:39.776 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:53:39.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:53:39.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:53:39.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:53:39.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:53:39.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:53:39.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:53:39.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:53:39.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:53:39.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:53:39.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:53:39.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:53:49.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:53:49.853 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:53:49.853 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:53:49.853 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:53:49.853 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:53:49.853 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:53:49.853 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:53:49.853 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:53:49.855 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:53:49.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:53:49.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:54:09.858 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:54:09.894 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:54:09.894 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:54:09.894 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:54:09.894 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:54:09.894 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:54:09.894 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:54:09.894 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:54:09.896 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:54:09.897 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:54:09.897 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:54:39.768 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:54:39.769 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:54:39.777 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:54:39.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:54:39.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:54:39.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:54:39.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:54:39.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:54:39.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:54:39.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:54:39.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:54:39.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:54:39.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:54:39.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:54:49.814 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:54:49.848 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:54:49.848 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:54:49.848 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:54:49.848 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:54:49.848 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:54:49.848 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:54:49.848 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:54:49.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:54:49.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:54:49.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:55:09.852 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:55:09.884 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:55:09.884 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:55:09.884 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:55:09.884 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:55:09.884 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:55:09.884 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:55:09.884 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:55:09.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:55:09.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:55:09.887 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:55:39.768 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:55:39.769 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:55:39.776 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:55:39.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:55:39.809 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:55:39.809 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:55:39.809 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:55:39.809 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:55:39.809 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:55:39.809 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:55:39.809 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:55:39.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:55:39.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:55:39.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:55:49.812 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:55:49.854 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:55:49.854 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:55:49.854 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:55:49.854 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:55:49.854 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:55:49.854 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:55:49.854 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:55:49.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:55:49.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:55:49.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:56:09.857 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:56:09.893 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:56:09.893 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:56:09.893 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:56:09.893 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:56:09.893 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:56:09.893 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:56:09.893 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:56:09.895 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:56:09.896 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:56:09.896 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:56:39.768 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:56:39.769 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:56:39.776 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:56:39.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:56:39.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:56:39.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:56:39.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:56:39.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:56:39.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:56:39.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:56:39.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:56:39.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:56:39.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:56:39.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:56:49.815 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:56:49.852 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:56:49.852 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:56:49.852 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:56:49.852 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:56:49.852 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:56:49.852 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:56:49.852 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:56:49.854 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:56:49.854 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:56:49.855 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:57:09.855 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:57:09.889 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:57:09.889 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:57:09.889 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:57:09.889 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:57:09.889 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:57:09.889 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:57:09.889 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:57:09.890 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:57:09.891 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:57:09.891 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:57:39.772 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:57:39.773 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:57:39.781 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:57:39.781 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:57:39.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:57:39.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:57:39.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:57:39.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:57:39.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:57:39.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:57:39.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:57:39.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:57:39.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:57:39.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:57:49.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:57:49.853 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:57:49.853 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:57:49.853 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:57:49.853 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:57:49.853 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:57:49.853 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:57:49.853 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:57:49.855 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:57:49.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:57:49.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:58:09.857 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:58:09.900 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:58:09.900 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:58:09.900 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:58:09.900 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:58:09.900 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:58:09.900 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:58:09.900 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:58:09.904 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:58:09.904 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:58:09.905 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:58:39.776 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:58:39.777 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:58:39.784 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:58:39.784 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:58:39.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:58:39.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:58:39.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:58:39.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:58:39.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:58:39.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:58:39.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:58:39.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:58:39.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:58:39.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:58:49.822 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:58:49.856 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:58:49.856 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:58:49.856 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:58:49.856 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:58:49.856 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:58:49.856 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:58:49.856 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:58:49.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:58:49.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:58:49.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:59:09.859 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:59:09.892 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:59:09.892 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:59:09.892 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:59:09.892 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:59:09.892 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:59:09.892 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:59:09.892 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:59:09.894 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:59:09.894 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 06:59:09.895 139328 ERROR oslo_service.periodic_task 
>2018-06-25 06:59:39.776 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:59:39.777 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 06:59:39.784 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 06:59:39.784 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:59:39.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:59:39.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:59:39.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:59:39.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:59:39.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:59:39.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:59:39.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:59:39.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:59:39.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:59:39.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 06:59:49.823 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 06:59:49.857 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 06:59:49.857 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 06:59:49.857 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 06:59:49.857 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 06:59:49.857 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 06:59:49.857 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 06:59:49.857 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 06:59:49.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 06:59:49.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 06:59:49.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:00:09.860 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:00:09.900 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:00:09.900 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:00:09.900 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:00:09.900 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:00:09.900 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:00:09.900 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:00:09.900 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:00:09.902 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:00:09.902 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:00:09.903 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:00:39.774 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:00:39.775 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:00:39.782 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:00:39.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:00:39.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:00:39.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:00:39.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:00:39.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:00:39.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:00:39.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:00:39.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:00:39.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:00:39.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:00:39.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:00:49.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:00:49.853 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:00:49.853 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:00:49.853 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:00:49.853 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:00:49.853 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:00:49.853 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:00:49.853 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:00:49.855 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:00:49.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:00:49.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:01:09.856 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:01:09.894 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:01:09.894 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:01:09.894 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:01:09.894 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:01:09.894 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:01:09.894 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:01:09.894 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:01:09.896 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:01:09.897 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:01:09.897 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:01:39.779 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:01:39.780 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:01:39.788 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:01:39.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:01:39.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:01:39.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:01:39.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:01:39.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:01:39.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:01:39.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:01:39.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:01:39.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:01:39.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:01:39.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:01:49.831 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:01:49.866 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:01:49.866 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:01:49.866 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:01:49.866 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:01:49.866 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:01:49.866 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:01:49.866 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:01:49.868 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:01:49.868 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:01:49.868 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:02:09.869 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:02:09.907 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:02:09.907 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:02:09.907 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:02:09.907 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:02:09.907 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:02:09.907 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:02:09.907 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:02:09.910 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:02:09.910 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.122 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:02:09.911 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:02:39.782 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:02:39.783 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:02:39.796 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:02:39.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:02:39.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:02:39.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:02:39.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:02:39.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:02:39.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:02:39.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:02:39.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:02:39.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:02:39.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:02:39.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:02:49.834 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:02:49.876 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:02:49.876 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:02:49.876 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:02:49.876 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:02:49.876 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:02:49.876 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:02:49.876 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:02:49.878 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:02:49.878 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:02:49.878 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:03:09.879 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:03:09.914 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:03:09.914 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:03:09.914 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:03:09.914 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:03:09.914 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:03:09.914 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:03:09.914 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:03:09.916 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:03:09.917 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:03:09.917 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:03:39.778 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:03:39.779 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:03:39.786 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:03:39.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:03:39.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:03:39.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:03:39.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:03:39.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:03:39.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:03:39.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:03:39.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:03:39.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:03:39.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:03:39.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:03:49.822 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:03:49.856 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:03:49.856 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:03:49.856 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:03:49.856 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:03:49.856 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:03:49.856 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:03:49.856 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:03:49.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:03:49.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:03:49.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:04:09.859 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:04:09.894 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:04:09.894 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:04:09.894 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:04:09.894 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:04:09.894 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:04:09.894 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:04:09.894 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:04:09.896 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:04:09.897 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:04:09.897 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:04:39.779 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:04:39.780 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:04:39.786 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:04:39.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:04:39.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:04:39.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:04:39.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:04:39.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:04:39.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:04:39.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:04:39.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:04:39.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:04:39.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:04:39.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:04:49.821 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:04:49.859 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:04:49.859 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:04:49.859 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:04:49.859 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:04:49.859 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:04:49.859 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:04:49.859 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:04:49.861 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:04:49.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:04:49.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:05:09.863 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:05:09.906 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:05:09.906 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:05:09.906 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:05:09.906 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:05:09.906 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:05:09.906 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:05:09.906 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:05:09.908 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:05:09.908 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:05:09.909 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:05:39.780 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:05:39.781 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:05:39.791 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:05:39.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:05:39.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:05:39.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:05:39.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:05:39.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:05:39.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:05:39.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:05:39.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:05:39.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:05:39.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.032 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:05:39.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:05:49.252 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume reschedule parameters: True retry: {u'num_attempts': 1, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1075
>2018-06-25 07:05:49.266 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (a8d0fbee-64ed-432e-bbae-682afccdb768) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-25 07:05:49.269 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (28b962bc-43a8-4a00-aa04-839e74df9f23) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:05:49.286 139328 DEBUG oslo_db.sqlalchemy.engines [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 07:05:49.315 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (28b962bc-43a8-4a00-aa04-839e74df9f23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:05:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:05:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:05:49.318 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51435dd5-7102-48ba-bf23-bd79bb19457a) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:05:49.319 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51435dd5-7102-48ba-bf23-bd79bb19457a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:05:49.321 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8f91cbad-a2d3-4a2e-b803-a83b004b87fe) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:05:49.322 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8f91cbad-a2d3-4a2e-b803-a83b004b87fe) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89', 'type': 'raw', 'volume_id': '99a0409d-7da9-40d8-b9ef-5a53003a6e89'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:05:49.324 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (44e8acc0-6197-491d-8a45-56836706fdb0) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:05:49.344 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (44e8acc0-6197-491d-8a45-56836706fdb0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:05:49.346 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1b8de662-88f5-46d3-956b-0da813fe9b71) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:05:49.347 139328 INFO cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89'}
>2018-06-25 07:05:49.347 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] creating volume 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89' create_volume /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:655
>2018-06-25 07:05:49.348 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:05:49.380 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:05:49.380 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:05:49.380 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:05:49.380 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:05:49.380 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:05:49.380 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:05:49.380 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:05:49.382 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:05:49.382 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:05:49.383 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:05:49.825 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:05:49.860 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:05:49.860 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:05:49.860 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:05:49.860 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:05:49.860 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:05:49.860 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:05:49.860 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:05:49.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:05:49.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:05:49.863 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:05:59.383 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:05:59.416 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:05:59.416 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:05:59.416 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:05:59.416 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:05:59.416 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:05:59.416 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:05:59.416 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:05:59.418 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:05:59.418 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:05:59.419 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:06:09.863 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:09.901 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:09.901 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:09.901 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:09.901 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:09.901 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:09.901 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:09.901 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:09.903 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:09.904 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:06:09.904 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:06:19.419 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:19.451 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:19.451 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:19.451 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:19.451 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:19.451 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:19.451 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:19.451 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:19.453 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:19.453 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:19.459 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1b8de662-88f5-46d3-956b-0da813fe9b71) transitioned into state 'FAILURE' from state 'RUNNING'
>5 predecessors (most recent first):
>  Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:05:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:05:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796f9c6790>}, 'provides': None}
>  |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:05:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:05:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(99a0409d-7da9-40d8-b9ef-5a53003a6e89),volume_id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7f796f9c6790>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89', 'type': 'raw', 'volume_id': '99a0409d-7da9-40d8-b9ef-5a53003a6e89'}}
>     |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:05:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:05:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'dde50b62-92b8-47a9-bbaf-6579fa39d69a', u'5fd520e5-826c-40f0-a931-58ab2e26eddd'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 1, u'display_name': u'asdf'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'dde50b62-92b8-47a9-bbaf-6579fa39d69a', u'5fd520e5-826c-40f0-a931-58ab2e26eddd'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 1, u'display_name': u'asdf'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'size': 1, u'display_name': u'asdf', u'display_description': None, u'cluster_name': None, u'name_id': u'99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'volume_admin_metadata': [], u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'bootable': False, u'created_at': u'2018-06-25T07:05:48.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'group_id': None}, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7f796f9c6790>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(99a0409d-7da9-40d8-b9ef-5a53003a6e89),volume_id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None}
>        |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:05:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:05:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796f9c6790>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:05:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:05:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)}
>           |__Flow 'volume_create_manager': VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     result = task.execute(**arguments)
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     ret = self.driver.create_volume(volume)
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     with RADOSClient(self) as client:
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     return _do_conn(pool, remote, timeout)
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     return r.call(f, *args, **kwargs)
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     raise attempt.get()
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:06:19.459 139328 ERROR cinder.volume.manager 
>2018-06-25 07:06:19.464 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1b8de662-88f5-46d3-956b-0da813fe9b71) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:19.466 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1b8de662-88f5-46d3-956b-0da813fe9b71) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:06:19.468 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (44e8acc0-6197-491d-8a45-56836706fdb0) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:19.469 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (44e8acc0-6197-491d-8a45-56836706fdb0) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:06:19.471 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8f91cbad-a2d3-4a2e-b803-a83b004b87fe) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:19.473 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8f91cbad-a2d3-4a2e-b803-a83b004b87fe) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:06:19.475 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51435dd5-7102-48ba-bf23-bd79bb19457a) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:19.476 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Updating volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89 with {'host': None, 'scheduled_at': datetime.datetime(2018, 6, 25, 7, 6, 19, 476414)}. _pre_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:121
>2018-06-25 07:06:19.482 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89: re-scheduling SchedulerAPI.create_volume attempt 1 due to Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster. _reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:148
>2018-06-25 07:06:19.498 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89: re-scheduled _post_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:160
>2018-06-25 07:06:19.499 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:19.537 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:19.537 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:19.537 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:19.537 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:19.537 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:19.537 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:19.537 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:19.540 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:19.541 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:19.541 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:06:19.636 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume reschedule parameters: True retry: {u'num_attempts': 2, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1075
>2018-06-25 07:06:19.650 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (0b68f81a-0868-4286-b194-69110d984672) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-25 07:06:19.653 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9285ba8b-6783-41e8-9b70-7e2235670086) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:19.671 139328 DEBUG oslo_db.sqlalchemy.engines [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 07:06:19.693 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9285ba8b-6783-41e8-9b70-7e2235670086) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:19Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:06:19.696 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (18e0e72e-3f56-4748-a5b6-9e0a8fa84747) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:19.698 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (18e0e72e-3f56-4748-a5b6-9e0a8fa84747) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:06:19.700 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5ee8e67e-be36-4d8c-8a55-391f37174927) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:19.701 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5ee8e67e-be36-4d8c-8a55-391f37174927) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89', 'type': 'raw', 'volume_id': '99a0409d-7da9-40d8-b9ef-5a53003a6e89'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:06:19.703 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (34ca840d-43c0-4acf-be64-43c7e72f3c05) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:19.725 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (34ca840d-43c0-4acf-be64-43c7e72f3c05) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:06:19.727 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b10255fb-0fcd-4a52-be44-6d8f74005995) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:19.728 139328 INFO cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89'}
>2018-06-25 07:06:19.728 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] creating volume 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89' create_volume /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:655
>2018-06-25 07:06:19.729 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:19.759 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:19.759 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:19.759 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:19.759 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:19.759 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:19.759 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:19.759 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:19.761 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:19.761 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 0.032 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:19.762 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:06:29.542 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:29.575 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:29.575 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:29.575 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:29.575 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:29.575 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:29.575 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:29.575 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:29.577 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:29.577 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:29.577 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:06:29.762 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:29.793 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:29.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:29.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:29.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:29.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:29.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:29.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:29.795 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:29.796 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 10.067 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:29.796 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:06:39.776 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:06:39.777 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:06:39.790 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:06:39.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:39.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:39.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:39.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:39.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:39.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:39.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:39.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:39.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:39.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.044 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:39.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:06:49.578 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:49.610 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:49.610 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:49.610 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:49.610 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:49.610 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:49.610 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:49.610 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:49.612 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:49.612 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:49.614 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51435dd5-7102-48ba-bf23-bd79bb19457a) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
>2018-06-25 07:06:49.615 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (28b962bc-43a8-4a00-aa04-839e74df9f23) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:49.617 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (28b962bc-43a8-4a00-aa04-839e74df9f23) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:06:49.619 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (a8d0fbee-64ed-432e-bbae-682afccdb768) transitioned into state 'REVERTED' from state 'RUNNING'
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "<string>", line 2, in create_volume
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     _run_flow()
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     flow_engine.run()
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     failures[0].reraise()
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     six.reraise(*self._exc_info)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     ret = self.driver.create_volume(volume)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     with RADOSClient(self) as client:
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     return _do_conn(pool, remote, timeout)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     return r.call(f, *args, **kwargs)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     raise attempt.get()
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:06:49.620 139328 ERROR oslo_messaging.rpc.server 
>2018-06-25 07:06:49.797 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:49.828 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:49.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:49.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:49.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:49.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:49.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:49.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:49.830 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:49.830 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 30.102 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:49.836 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b10255fb-0fcd-4a52-be44-6d8f74005995) transitioned into state 'FAILURE' from state 'RUNNING'
>5 predecessors (most recent first):
>  Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:19Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796fafe950>}, 'provides': None}
>  |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:19Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(99a0409d-7da9-40d8-b9ef-5a53003a6e89),volume_id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7f796fafe950>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89', 'type': 'raw', 'volume_id': '99a0409d-7da9-40d8-b9ef-5a53003a6e89'}}
>     |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:19Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'dde50b62-92b8-47a9-bbaf-6579fa39d69a', u'5fd520e5-826c-40f0-a931-58ab2e26eddd'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 1, u'display_name': u'asdf'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'dde50b62-92b8-47a9-bbaf-6579fa39d69a', u'5fd520e5-826c-40f0-a931-58ab2e26eddd'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 1, u'display_name': u'asdf'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'size': 1, u'display_name': u'asdf', u'display_description': None, u'cluster_name': None, u'name_id': u'99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'volume_admin_metadata': [], u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'admin_metadata': {}, u'host': None, u'glance_metadata': {}, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'bootable': False, u'created_at': u'2018-06-25T07:05:48.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'group_id': None}, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 2, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7f796fafe950>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(99a0409d-7da9-40d8-b9ef-5a53003a6e89),volume_id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None}
>        |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:19Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796fafe950>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:19Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:19Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)}
>           |__Flow 'volume_create_manager': VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     result = task.execute(**arguments)
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     ret = self.driver.create_volume(volume)
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     with RADOSClient(self) as client:
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     return _do_conn(pool, remote, timeout)
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     return r.call(f, *args, **kwargs)
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     raise attempt.get()
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:06:49.836 139328 ERROR cinder.volume.manager 
>2018-06-25 07:06:49.840 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b10255fb-0fcd-4a52-be44-6d8f74005995) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:49.842 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b10255fb-0fcd-4a52-be44-6d8f74005995) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:06:49.844 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (34ca840d-43c0-4acf-be64-43c7e72f3c05) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:49.845 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (34ca840d-43c0-4acf-be64-43c7e72f3c05) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:06:49.847 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5ee8e67e-be36-4d8c-8a55-391f37174927) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:49.848 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5ee8e67e-be36-4d8c-8a55-391f37174927) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:06:49.850 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (18e0e72e-3f56-4748-a5b6-9e0a8fa84747) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:49.851 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Updating volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89 with {'host': None, 'scheduled_at': datetime.datetime(2018, 6, 25, 7, 6, 49, 851225)}. _pre_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:121
>2018-06-25 07:06:49.853 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:49.884 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:49.884 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:49.884 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:49.884 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:49.884 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:49.884 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:49.884 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:49.885 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:49.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.095 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:49.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:06:49.890 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89: re-scheduling SchedulerAPI.create_volume attempt 2 due to Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster. _reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:148
>2018-06-25 07:06:49.904 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89: re-scheduled _post_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:160
>2018-06-25 07:06:49.905 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:49.936 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:49.936 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:49.936 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:49.936 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:49.936 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:49.936 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:49.936 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:49.939 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:49.939 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:49.940 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:06:50.014 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume reschedule parameters: True retry: {u'num_attempts': 3, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1075
>2018-06-25 07:06:50.029 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (755fa346-820b-4cfb-ae30-b74ce36645b3) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-25 07:06:50.032 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5eb0850e-187d-41e9-b15a-32060740d297) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:50.048 139328 DEBUG oslo_db.sqlalchemy.engines [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 07:06:50.069 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5eb0850e-187d-41e9-b15a-32060740d297) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:06:50.071 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9fc62cda-2645-447a-85a5-4e53d2bb4f69) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:50.073 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9fc62cda-2645-447a-85a5-4e53d2bb4f69) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:06:50.075 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5d790c61-63f5-42ae-8072-eb855799d5e6) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:50.076 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5d790c61-63f5-42ae-8072-eb855799d5e6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89', 'type': 'raw', 'volume_id': '99a0409d-7da9-40d8-b9ef-5a53003a6e89'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:06:50.078 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9daf853f-f751-4742-af5f-b89e6e3e2202) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:50.098 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9daf853f-f751-4742-af5f-b89e6e3e2202) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 07:06:50.100 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7d0414ef-bc7b-4b21-a4f6-46c4dbe725ac) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:06:50.101 139328 INFO cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89: being created as raw with specification: {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89'}
>2018-06-25 07:06:50.101 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] creating volume 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89' create_volume /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:655
>2018-06-25 07:06:50.102 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:50.133 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:50.133 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:50.133 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:50.133 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:50.133 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:50.133 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:50.133 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:50.135 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:50.135 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:50.135 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:06:59.941 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:06:59.975 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:59.975 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:06:59.975 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:06:59.975 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:06:59.975 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:06:59.975 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:06:59.975 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:06:59.977 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:06:59.977 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:06:59.977 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:07:00.136 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:07:00.167 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:00.167 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:07:00.167 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:07:00.167 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:07:00.167 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:07:00.167 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:00.167 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:07:00.168 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:07:00.169 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 10.067 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:07:00.169 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:07:09.886 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:07:09.920 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:09.920 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:07:09.920 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:07:09.920 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:07:09.920 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:07:09.920 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:09.920 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:07:09.923 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:07:09.924 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.133 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:07:09.924 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:07:19.978 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:07:20.012 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:20.012 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:07:20.012 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:07:20.012 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:07:20.012 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:07:20.012 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:20.012 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:07:20.014 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:07:20.015 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:07:20.016 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (18e0e72e-3f56-4748-a5b6-9e0a8fa84747) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
>2018-06-25 07:07:20.018 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9285ba8b-6783-41e8-9b70-7e2235670086) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:07:20.020 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9285ba8b-6783-41e8-9b70-7e2235670086) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:07:20.022 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (0b68f81a-0868-4286-b194-69110d984672) transitioned into state 'REVERTED' from state 'RUNNING'
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "<string>", line 2, in create_volume
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     _run_flow()
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     flow_engine.run()
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     failures[0].reraise()
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     six.reraise(*self._exc_info)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     ret = self.driver.create_volume(volume)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     with RADOSClient(self) as client:
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     return _do_conn(pool, remote, timeout)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     return r.call(f, *args, **kwargs)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     raise attempt.get()
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:07:20.023 139328 ERROR oslo_messaging.rpc.server 
>2018-06-25 07:07:20.170 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:07:20.201 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:20.201 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:07:20.201 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:07:20.201 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:07:20.201 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:07:20.201 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:20.201 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:07:20.203 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:07:20.203 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 30.101 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:07:20.208 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7d0414ef-bc7b-4b21-a4f6-46c4dbe725ac) transitioned into state 'FAILURE' from state 'RUNNING'
>5 predecessors (most recent first):
>  Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796f7fdb90>}, 'provides': None}
>  |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(99a0409d-7da9-40d8-b9ef-5a53003a6e89),volume_id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7f796f7fdb90>}, 'provides': {'status': u'creating', 'volume_size': 1, 'volume_name': 'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89', 'type': 'raw', 'volume_id': '99a0409d-7da9-40d8-b9ef-5a53003a6e89'}}
>     |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'dde50b62-92b8-47a9-bbaf-6579fa39d69a', u'5fd520e5-826c-40f0-a931-58ab2e26eddd'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 1, u'display_name': u'asdf'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'dde50b62-92b8-47a9-bbaf-6579fa39d69a', u'5fd520e5-826c-40f0-a931-58ab2e26eddd'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 1, u'display_name': u'asdf'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'size': 1, u'display_name': u'asdf', u'display_description': None, u'cluster_name': None, u'name_id': u'99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'volume_admin_metadata': [], u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'admin_metadata': {}, u'host': None, u'glance_metadata': {}, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'bootable': False, u'created_at': u'2018-06-25T07:05:48.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'99a0409d-7da9-40d8-b9ef-5a53003a6e89', u'group_id': None}, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 3, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']}, u'metadata': {}, u'resource_type': {}, u'size': 1}, 'context': <cinder.context.RequestContext object at 0x7f796f7fdb90>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(99a0409d-7da9-40d8-b9ef-5a53003a6e89),volume_id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None}
>        |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7f796f7fdb90>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T07:05:48Z,deleted=False,deleted_at=None,display_description=None,display_name='asdf',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=99a0409d-7da9-40d8-b9ef-5a53003a6e89,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T07:06:49Z,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T07:06:49Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)}
>           |__Flow 'volume_create_manager': VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     result = task.execute(**arguments)
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     ret = self.driver.create_volume(volume)
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     with RADOSClient(self) as client:
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     return _do_conn(pool, remote, timeout)
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     return r.call(f, *args, **kwargs)
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     raise attempt.get()
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:07:20.208 139328 ERROR cinder.volume.manager 
>2018-06-25 07:07:20.213 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7d0414ef-bc7b-4b21-a4f6-46c4dbe725ac) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:07:20.214 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7d0414ef-bc7b-4b21-a4f6-46c4dbe725ac) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:07:20.216 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9daf853f-f751-4742-af5f-b89e6e3e2202) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:07:20.218 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9daf853f-f751-4742-af5f-b89e6e3e2202) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:07:20.220 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5d790c61-63f5-42ae-8072-eb855799d5e6) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:07:20.221 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5d790c61-63f5-42ae-8072-eb855799d5e6) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:07:20.223 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9fc62cda-2645-447a-85a5-4e53d2bb4f69) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:07:20.224 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Updating volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89 with {'host': None, 'scheduled_at': datetime.datetime(2018, 6, 25, 7, 7, 20, 224099)}. _pre_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:121
>2018-06-25 07:07:20.231 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89: re-scheduling SchedulerAPI.create_volume attempt 3 due to Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster. _reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:148
>2018-06-25 07:07:20.252 139328 DEBUG cinder.volume.flows.manager.create_volume [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume 99a0409d-7da9-40d8-b9ef-5a53003a6e89: re-scheduled _post_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:160
>2018-06-25 07:07:20.253 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:07:20.285 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:20.285 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:07:20.285 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:07:20.285 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:07:20.285 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:07:20.285 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:20.285 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:07:20.287 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:07:20.287 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:07:20.288 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:07:30.288 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:07:30.320 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:30.320 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:07:30.320 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:07:30.320 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:07:30.320 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:07:30.320 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:30.320 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:07:30.322 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:07:30.322 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:07:30.323 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:07:39.775 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:07:39.776 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:07:39.782 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:07:39.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:07:39.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:39.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:07:39.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:07:39.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:07:39.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:07:39.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:39.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:07:39.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:07:39.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:07:39.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:07:49.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:07:49.854 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:49.854 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:07:49.854 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:07:49.854 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:07:49.854 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:07:49.854 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:49.854 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:07:49.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:07:49.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:07:49.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:07:50.323 139328 DEBUG cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:07:50.354 139328 ERROR cinder.volume.drivers.rbd [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:50.354 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:07:50.354 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:07:50.354 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:07:50.354 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:07:50.354 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:07:50.354 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:07:50.356 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:07:50.356 139328 DEBUG cinder.utils [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 30.103 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:07:50.358 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9fc62cda-2645-447a-85a5-4e53d2bb4f69) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
>2018-06-25 07:07:50.360 139328 DEBUG cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5eb0850e-187d-41e9-b15a-32060740d297) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 07:07:50.361 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5eb0850e-187d-41e9-b15a-32060740d297) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 07:07:50.363 139328 WARNING cinder.volume.manager [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (755fa346-820b-4cfb-ae30-b74ce36645b3) transitioned into state 'REVERTED' from state 'RUNNING'
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server [req-2aa61f79-c724-44b5-8769-c34f952663a2 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Exception during message handling: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "<string>", line 2, in create_volume
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     _run_flow()
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     flow_engine.run()
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     failures[0].reraise()
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     six.reraise(*self._exc_info)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 942, in execute
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     model_update = self._create_raw_volume(volume, **volume_spec)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 910, in _create_raw_volume
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     ret = self.driver.create_volume(volume)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 661, in create_volume
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     with RADOSClient(self) as client:
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     return _do_conn(pool, remote, timeout)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     return r.call(f, *args, **kwargs)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     raise attempt.get()
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:07:50.364 139328 ERROR oslo_messaging.rpc.server 
>2018-06-25 07:08:09.858 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:08:09.891 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:08:09.891 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:08:09.891 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:08:09.891 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:08:09.891 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:08:09.891 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:08:09.891 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:08:09.893 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:08:09.893 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:08:09.893 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:08:39.774 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:08:39.775 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:08:39.787 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:08:39.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:08:39.828 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:08:39.828 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:08:39.828 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:08:39.828 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:08:39.828 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:08:39.828 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:08:39.828 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:08:39.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:08:39.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:08:39.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:08:49.831 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:08:49.863 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:08:49.863 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:08:49.863 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:08:49.863 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:08:49.863 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:08:49.863 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:08:49.863 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:08:49.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:08:49.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:08:49.866 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:09:09.866 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:09:09.900 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:09:09.900 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:09:09.900 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:09:09.900 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:09:09.900 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:09:09.900 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:09:09.900 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:09:09.902 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:09:09.903 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:09:09.903 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:09:39.774 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:09:39.775 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:09:39.788 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:09:39.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:09:39.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:09:39.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:09:39.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:09:39.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:09:39.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:09:39.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:09:39.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:09:39.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:09:39.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:09:39.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:09:49.826 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:09:49.859 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:09:49.859 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:09:49.859 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:09:49.859 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:09:49.859 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:09:49.859 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:09:49.859 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:09:49.861 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:09:49.861 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:09:49.861 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:10:09.862 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:10:09.899 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:10:09.899 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:10:09.899 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:10:09.899 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:10:09.899 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:10:09.899 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:10:09.899 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:10:09.902 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:10:09.902 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:10:09.902 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:10:39.774 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:10:39.774 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:10:39.781 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:10:39.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:10:39.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:10:39.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:10:39.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:10:39.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:10:39.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:10:39.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:10:39.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:10:39.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:10:39.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:10:39.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:10:49.817 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:10:49.850 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:10:49.850 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:10:49.850 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:10:49.850 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:10:49.850 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:10:49.850 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:10:49.850 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:10:49.852 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:10:49.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:10:49.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:11:09.853 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:11:09.886 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:11:09.886 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:11:09.886 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:11:09.886 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:11:09.886 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:11:09.886 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:11:09.886 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:11:09.888 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:11:09.888 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:11:09.889 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:11:39.770 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:11:39.770 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:11:39.777 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:11:39.777 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:11:39.811 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:11:39.811 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:11:39.811 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:11:39.811 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:11:39.811 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:11:39.811 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:11:39.811 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:11:39.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:11:39.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:11:39.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:11:49.815 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:11:49.854 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:11:49.854 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:11:49.854 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:11:49.854 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:11:49.854 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:11:49.854 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:11:49.854 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:11:49.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:11:49.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:11:49.857 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:12:09.858 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:12:09.892 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:12:09.892 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:12:09.892 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:12:09.892 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:12:09.892 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:12:09.892 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:12:09.892 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:12:09.894 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:12:09.894 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.116 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:12:09.894 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:12:39.766 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:12:39.766 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:12:39.772 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:12:39.773 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:12:39.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:12:39.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:12:39.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:12:39.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:12:39.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:12:39.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:12:39.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:12:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:12:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:12:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:12:49.807 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:12:49.840 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:12:49.840 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:12:49.840 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:12:49.840 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:12:49.840 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:12:49.840 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:12:49.840 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:12:49.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:12:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:12:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:13:09.843 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:13:09.877 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:13:09.877 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:13:09.877 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:13:09.877 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:13:09.877 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:13:09.877 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:13:09.877 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:13:09.879 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:13:09.880 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:13:09.880 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:13:39.762 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:13:39.763 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:13:39.771 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:13:39.772 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:13:39.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:13:39.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:13:39.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:13:39.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:13:39.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:13:39.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:13:39.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:13:39.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:13:39.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.047 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:13:39.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:13:49.819 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:13:49.857 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:13:49.857 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:13:49.857 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:13:49.857 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:13:49.857 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:13:49.857 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:13:49.857 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:13:49.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:13:49.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:13:49.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:14:09.861 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:14:09.895 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:14:09.895 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:14:09.895 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:14:09.895 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:14:09.895 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:14:09.895 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:14:09.895 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:14:09.898 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:14:09.899 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.126 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:14:09.899 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:14:39.760 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:14:39.761 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:14:39.768 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:14:39.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:14:39.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:14:39.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:14:39.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:14:39.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:14:39.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:14:39.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:14:39.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:14:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:14:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:14:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:14:49.807 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:14:49.840 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:14:49.840 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:14:49.840 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:14:49.840 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:14:49.840 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:14:49.840 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:14:49.840 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:14:49.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:14:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:14:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:15:09.844 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:15:09.877 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:15:09.877 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:15:09.877 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:15:09.877 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:15:09.877 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:15:09.877 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:15:09.877 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:15:09.879 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:15:09.879 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:15:09.880 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:15:39.761 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:15:39.761 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:15:39.768 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:15:39.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:15:39.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:15:39.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:15:39.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:15:39.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:15:39.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:15:39.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:15:39.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:15:39.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:15:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:15:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:15:49.806 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:15:49.840 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:15:49.840 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:15:49.840 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:15:49.840 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:15:49.840 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:15:49.840 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:15:49.840 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:15:49.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:15:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:15:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:16:09.844 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:16:09.876 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:16:09.876 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:16:09.876 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:16:09.876 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:16:09.876 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:16:09.876 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:16:09.876 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:16:09.877 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:16:09.878 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:16:09.878 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:16:39.759 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:16:39.760 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:16:39.768 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:16:39.769 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:16:39.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:16:39.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:16:39.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:16:39.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:16:39.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:16:39.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:16:39.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:16:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:16:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:16:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:16:49.806 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:16:49.843 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:16:49.843 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:16:49.843 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:16:49.843 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:16:49.843 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:16:49.843 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:16:49.843 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:16:49.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:16:49.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:16:49.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:17:09.847 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:17:09.881 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:17:09.881 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:17:09.881 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:17:09.881 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:17:09.881 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:17:09.881 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:17:09.881 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:17:09.883 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:17:09.884 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:17:09.884 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:17:39.755 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:17:39.756 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:17:39.763 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:17:39.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:17:39.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:17:39.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:17:39.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:17:39.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:17:39.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:17:39.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:17:39.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:17:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:17:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:17:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:17:49.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:17:49.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:17:49.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:17:49.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:17:49.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:17:49.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:17:49.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:17:49.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:17:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:17:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:17:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:18:09.835 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:18:09.876 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:18:09.876 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:18:09.876 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:18:09.876 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:18:09.876 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:18:09.876 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:18:09.876 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:18:09.878 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:18:09.879 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:18:09.879 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:18:39.760 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:18:39.761 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:18:39.768 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:18:39.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:18:39.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:18:39.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:18:39.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:18:39.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:18:39.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:18:39.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:18:39.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:18:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:18:39.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:18:39.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:18:49.807 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:18:49.842 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:18:49.842 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:18:49.842 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:18:49.842 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:18:49.842 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:18:49.842 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:18:49.842 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:18:49.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:18:49.845 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:18:49.845 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:19:09.846 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:19:09.879 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:19:09.879 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:19:09.879 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:19:09.879 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:19:09.879 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:19:09.879 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:19:09.879 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:19:09.881 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:19:09.881 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:19:09.882 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:19:39.763 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:19:39.764 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:19:39.771 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:19:39.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:19:39.804 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:19:39.804 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:19:39.804 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:19:39.804 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:19:39.804 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:19:39.804 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:19:39.804 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:19:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:19:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:19:39.807 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:19:49.807 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:19:49.841 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:19:49.841 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:19:49.841 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:19:49.841 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:19:49.841 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:19:49.841 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:19:49.841 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:19:49.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:19:49.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:19:49.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:20:09.845 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:20:09.879 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:20:09.879 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:20:09.879 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:20:09.879 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:20:09.879 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:20:09.879 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:20:09.879 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:20:09.881 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:20:09.881 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:20:09.882 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:20:39.763 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:20:39.763 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:20:39.770 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:20:39.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:20:39.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:20:39.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:20:39.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:20:39.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:20:39.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:20:39.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:20:39.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:20:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:20:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:20:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:20:49.806 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:20:49.840 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:20:49.840 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:20:49.840 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:20:49.840 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:20:49.840 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:20:49.840 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:20:49.840 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:20:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:20:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:20:49.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:21:09.844 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:21:09.878 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:21:09.878 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:21:09.878 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:21:09.878 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:21:09.878 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:21:09.878 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:21:09.878 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:21:09.879 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:21:09.880 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:21:09.880 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:21:39.761 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:21:39.762 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:21:39.769 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:21:39.770 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:21:39.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:21:39.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:21:39.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:21:39.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:21:39.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:21:39.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:21:39.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:21:39.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:21:39.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.042 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:21:39.813 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:21:49.814 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:21:49.847 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:21:49.847 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:21:49.847 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:21:49.847 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:21:49.847 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:21:49.847 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:21:49.847 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:21:49.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:21:49.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:21:49.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:22:09.850 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:22:09.885 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:22:09.885 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:22:09.885 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:22:09.885 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:22:09.885 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:22:09.885 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:22:09.885 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:22:09.888 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:22:09.888 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:22:09.888 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:22:39.760 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:22:39.761 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:22:39.767 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:22:39.768 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:22:39.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:22:39.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:22:39.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:22:39.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:22:39.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:22:39.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:22:39.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:22:39.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:22:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:22:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:22:49.805 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:22:49.839 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:22:49.839 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:22:49.839 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:22:49.839 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:22:49.839 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:22:49.839 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:22:49.839 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:22:49.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:22:49.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:22:49.842 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:23:09.842 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:23:09.883 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:23:09.883 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:23:09.883 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:23:09.883 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:23:09.883 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:23:09.883 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:23:09.883 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:23:09.885 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:23:09.886 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:23:09.886 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:23:39.757 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:23:39.758 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:23:39.766 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:23:39.767 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:23:39.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:23:39.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:23:39.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:23:39.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:23:39.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:23:39.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:23:39.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:23:39.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:23:39.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:23:39.803 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:23:49.804 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:23:49.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:23:49.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:23:49.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:23:49.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:23:49.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:23:49.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:23:49.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:23:49.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:23:49.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:23:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:24:09.840 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:24:09.875 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:24:09.875 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:24:09.875 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:24:09.875 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:24:09.875 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:24:09.875 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:24:09.875 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:24:09.878 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:24:09.878 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:24:09.878 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:24:39.760 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:24:39.760 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:24:39.770 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:24:39.771 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:24:39.802 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:24:39.802 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:24:39.802 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:24:39.802 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:24:39.802 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:24:39.802 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:24:39.802 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:24:39.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:24:39.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:24:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:24:49.805 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:24:49.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:24:49.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:24:49.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:24:49.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:24:49.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:24:49.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:24:49.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:24:49.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:24:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:24:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:25:09.840 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:25:09.874 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:25:09.874 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:25:09.874 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:25:09.874 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:25:09.874 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:25:09.874 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:25:09.874 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:25:09.876 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:25:09.876 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:25:09.877 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:25:39.758 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:25:39.759 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:25:39.765 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:25:39.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:25:39.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:25:39.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:25:39.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:25:39.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:25:39.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:25:39.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:25:39.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:25:39.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:25:39.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:25:39.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:25:49.799 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:25:49.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:25:49.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:25:49.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:25:49.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:25:49.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:25:49.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:25:49.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:25:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:25:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.068 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:25:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:26:09.835 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:26:09.871 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:26:09.871 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:26:09.871 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:26:09.871 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:26:09.871 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:26:09.871 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:26:09.871 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:26:09.873 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:26:09.873 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:26:09.873 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:26:39.755 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:26:39.756 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:26:39.763 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:26:39.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:26:39.799 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:26:39.799 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:26:39.799 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:26:39.799 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:26:39.799 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:26:39.799 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:26:39.799 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:26:39.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:26:39.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:26:39.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:26:49.803 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:26:49.837 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:26:49.837 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:26:49.837 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:26:49.837 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:26:49.837 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:26:49.837 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:26:49.837 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:26:49.839 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:26:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:26:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:27:09.840 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:27:09.875 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:27:09.875 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:27:09.875 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:27:09.875 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:27:09.875 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:27:09.875 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:27:09.875 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:27:09.877 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:27:09.877 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:27:09.878 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:27:39.750 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:27:39.751 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:27:39.757 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:27:39.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:27:39.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:27:39.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:27:39.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:27:39.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:27:39.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:27:39.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:27:39.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:27:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:27:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:27:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:27:49.796 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:27:49.829 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:27:49.829 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:27:49.829 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:27:49.829 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:27:49.829 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:27:49.829 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:27:49.829 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:27:49.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:27:49.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:27:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:28:09.832 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:28:09.870 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:28:09.870 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:28:09.870 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:28:09.870 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:28:09.870 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:28:09.870 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:28:09.870 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:28:09.872 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:28:09.873 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:28:09.873 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:28:39.755 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:28:39.755 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:28:39.761 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:28:39.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:28:39.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:28:39.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:28:39.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:28:39.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:28:39.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:28:39.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:28:39.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:28:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:28:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:28:39.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:28:49.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:28:49.834 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:28:49.834 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:28:49.834 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:28:49.834 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:28:49.834 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:28:49.834 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:28:49.834 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:28:49.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:28:49.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:28:49.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:29:09.837 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:29:09.870 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:29:09.870 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:29:09.870 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:29:09.870 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:29:09.870 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:29:09.870 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:29:09.870 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:29:09.872 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:29:09.872 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:29:09.872 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:29:39.754 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:29:39.755 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:29:39.761 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:29:39.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:29:39.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:29:39.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:29:39.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:29:39.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:29:39.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:29:39.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:29:39.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:29:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:29:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:29:39.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:29:49.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:29:49.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:29:49.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:29:49.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:29:49.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:29:49.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:29:49.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:29:49.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:29:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:29:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:29:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:30:09.833 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:30:09.867 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:30:09.867 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:30:09.867 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:30:09.867 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:30:09.867 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:30:09.867 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:30:09.867 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:30:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:30:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:30:09.870 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:30:39.751 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:30:39.752 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:30:39.758 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:30:39.759 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:30:39.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:30:39.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:30:39.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:30:39.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:30:39.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:30:39.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:30:39.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:30:39.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:30:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:30:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:30:49.793 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:30:49.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:30:49.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:30:49.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:30:49.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:30:49.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:30:49.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:30:49.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:30:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:30:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.067 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:30:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:31:09.827 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:31:09.860 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:31:09.860 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:31:09.860 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:31:09.860 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:31:09.860 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:31:09.860 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:31:09.860 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:31:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:31:09.863 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.104 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:31:09.863 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:31:39.754 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:31:39.755 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:31:39.761 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:31:39.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:31:39.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:31:39.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:31:39.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:31:39.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:31:39.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:31:39.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:31:39.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:31:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:31:39.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:31:39.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:31:49.799 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:31:49.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:31:49.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:31:49.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:31:49.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:31:49.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:31:49.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:31:49.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:31:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:31:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:31:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:32:09.834 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:32:09.868 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:32:09.868 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:32:09.868 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:32:09.868 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:32:09.868 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:32:09.868 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:32:09.868 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:32:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:32:09.870 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:32:09.870 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:32:39.752 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:32:39.753 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:32:39.764 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:32:39.765 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:32:39.803 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:32:39.803 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:32:39.803 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:32:39.803 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:32:39.803 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:32:39.803 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:32:39.803 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:32:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:32:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:32:39.806 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:32:49.806 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:32:49.838 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:32:49.838 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:32:49.838 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:32:49.838 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:32:49.838 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:32:49.838 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:32:49.838 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:32:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:32:49.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:32:49.841 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:33:09.841 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:33:09.876 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:33:09.876 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:33:09.876 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:33:09.876 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:33:09.876 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:33:09.876 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:33:09.876 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:33:09.878 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:33:09.879 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:33:09.879 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:33:39.750 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:33:39.751 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:33:39.758 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:33:39.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:33:39.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:33:39.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:33:39.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:33:39.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:33:39.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:33:39.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:33:39.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:33:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:33:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:33:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:33:49.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:33:49.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:33:49.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:33:49.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:33:49.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:33:49.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:33:49.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:33:49.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:33:49.831 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:33:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:33:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:34:09.833 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:34:09.869 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:34:09.869 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:34:09.869 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:34:09.869 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:34:09.869 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:34:09.869 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:34:09.869 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:34:09.871 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:34:09.872 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:34:09.872 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:34:39.753 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:34:39.754 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:34:39.761 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:34:39.762 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:34:39.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:34:39.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:34:39.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:34:39.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:34:39.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:34:39.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:34:39.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:34:39.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:34:39.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:34:39.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:34:49.801 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:34:49.833 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:34:49.833 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:34:49.833 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:34:49.833 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:34:49.833 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:34:49.833 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:34:49.833 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:34:49.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:34:49.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:34:49.835 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:35:09.836 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:35:09.870 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:35:09.870 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:35:09.870 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:35:09.870 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:35:09.870 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:35:09.870 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:35:09.870 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:35:09.872 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:35:09.873 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:35:09.873 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:35:39.754 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:35:39.755 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:35:39.762 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:35:39.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:35:39.795 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:35:39.795 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:35:39.795 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:35:39.795 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:35:39.795 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:35:39.795 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:35:39.795 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:35:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:35:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:35:39.798 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:35:49.798 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:35:49.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:35:49.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:35:49.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:35:49.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:35:49.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:35:49.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:35:49.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:35:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:35:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:35:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:36:09.834 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:36:09.868 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:36:09.868 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:36:09.868 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:36:09.868 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:36:09.868 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:36:09.868 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:36:09.868 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:36:09.870 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:36:09.870 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.107 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:36:09.871 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:36:39.752 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:36:39.753 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:36:39.760 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:36:39.761 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:36:39.800 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:36:39.800 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:36:39.800 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:36:39.800 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:36:39.800 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:36:39.800 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:36:39.800 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:36:39.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:36:39.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:36:39.802 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:36:49.803 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:36:49.842 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:36:49.842 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:36:49.842 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:36:49.842 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:36:49.842 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:36:49.842 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:36:49.842 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:36:49.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:36:49.844 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:36:49.845 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:37:09.845 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:37:09.879 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:37:09.879 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:37:09.879 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:37:09.879 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:37:09.879 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:37:09.879 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:37:09.879 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:37:09.881 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:37:09.882 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.12 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:37:09.882 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:37:39.753 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:37:39.754 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:38:39.752 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:38:39.753 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:38:39.762 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:38:39.763 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:38:39.794 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:38:39.794 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:38:39.794 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:38:39.794 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:38:39.794 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:38:39.794 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:38:39.794 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:38:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:38:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:38:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:38:49.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:38:49.830 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:38:49.830 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:38:49.830 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:38:49.830 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:38:49.830 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:38:49.830 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:38:49.830 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:38:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:38:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:38:49.832 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:39:09.833 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:39:09.864 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:39:09.864 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:39:09.864 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:39:09.864 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:39:09.864 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:39:09.864 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:39:09.864 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:39:09.866 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:39:09.867 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.104 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:39:09.867 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:39:39.748 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:39:39.749 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:39:39.756 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:39:39.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:39:39.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:39:39.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:39:39.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:39:39.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:39:39.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:39:39.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:39:39.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:39:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:39:39.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:39:39.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:39:49.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:39:49.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:39:49.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:39:49.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:39:49.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:39:49.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:39:49.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:39:49.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:39:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:39:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.068 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:39:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:40:09.826 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:40:09.860 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:40:09.860 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:40:09.860 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:40:09.860 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:40:09.860 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:40:09.860 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:40:09.860 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:40:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:40:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:40:09.863 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:40:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:40:39.745 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:40:39.757 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:40:39.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:40:39.796 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:40:39.796 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:40:39.796 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:40:39.796 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:40:39.796 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:40:39.796 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:40:39.796 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:40:39.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:40:39.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:40:39.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:40:49.800 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:40:49.832 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:40:49.832 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:40:49.832 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:40:49.832 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:40:49.832 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:40:49.832 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:40:49.832 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:40:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:40:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:40:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:41:09.835 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:41:09.870 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:41:09.870 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:41:09.870 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:41:09.870 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:41:09.870 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:41:09.870 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:41:09.870 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:41:09.873 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:41:09.874 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:41:09.874 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:41:39.746 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:41:39.747 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:41:39.754 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:41:39.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:41:39.792 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:41:39.792 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:41:39.792 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:41:39.792 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:41:39.792 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:41:39.792 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:41:39.792 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:41:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:41:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:41:39.795 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:41:49.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:41:49.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:41:49.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:41:49.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:41:49.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:41:49.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:41:49.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:41:49.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:41:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:41:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:41:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:42:09.829 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:42:09.870 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:42:09.870 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:42:09.870 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:42:09.870 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:42:09.870 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:42:09.870 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:42:09.870 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:42:09.872 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:42:09.873 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:42:09.873 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:42:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:42:39.745 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:42:39.752 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:42:39.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:42:39.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:42:39.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:42:39.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:42:39.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:42:39.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:42:39.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:42:39.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:42:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:42:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:42:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:42:49.794 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:42:49.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:42:49.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:42:49.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:42:49.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:42:49.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:42:49.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:42:49.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:42:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:42:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:42:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:43:09.829 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:43:09.862 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:43:09.862 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:43:09.862 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:43:09.862 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:43:09.862 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:43:09.862 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:43:09.862 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:43:09.864 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:43:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.112 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:43:09.865 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:43:39.746 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:43:39.747 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:43:39.754 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:43:39.754 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:43:39.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:43:39.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:43:39.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:43:39.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:43:39.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:43:39.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:43:39.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:43:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:43:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:43:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:43:49.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:43:49.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:43:49.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:43:49.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:43:49.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:43:49.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:43:49.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:43:49.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:43:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:43:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:43:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:44:09.827 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:44:09.861 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:44:09.861 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:44:09.861 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:44:09.861 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:44:09.861 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:44:09.861 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:44:09.861 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:44:09.863 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:44:09.864 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:44:09.864 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:44:39.745 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:44:39.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:44:39.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:44:39.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:44:39.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:44:39.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:44:39.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:44:39.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:44:39.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:44:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:44:39.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:44:39.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:44:49.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:44:49.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:44:49.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:44:49.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:44:49.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:44:49.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:44:49.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:44:49.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:44:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:44:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:44:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:45:09.827 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:45:09.861 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:45:09.861 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:45:09.861 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:45:09.861 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:45:09.861 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:45:09.861 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:45:09.861 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:45:09.864 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:45:09.864 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:45:09.865 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:45:39.746 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:45:39.747 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:45:39.753 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:45:39.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:45:39.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:45:39.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:45:39.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:45:39.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:45:39.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:45:39.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:45:39.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:45:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:45:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:45:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:45:49.789 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:45:49.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:45:49.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:45:49.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:45:49.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:45:49.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:45:49.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:45:49.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:45:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:45:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.069 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:45:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:46:09.824 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:46:09.861 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:46:09.861 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:46:09.861 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:46:09.861 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:46:09.861 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:46:09.861 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:46:09.861 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:46:09.863 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:46:09.863 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:46:09.863 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:46:39.745 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:46:39.746 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:46:39.753 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:46:39.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:46:39.787 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:46:39.787 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:46:39.787 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:46:39.787 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:46:39.787 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:46:39.787 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:46:39.787 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:46:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:46:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:46:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:46:49.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:46:49.841 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:46:49.841 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:46:49.841 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:46:49.841 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:46:49.841 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:46:49.841 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:46:49.841 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:46:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:46:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.089 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:46:49.843 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:47:09.844 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:47:09.880 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:47:09.880 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:47:09.880 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:47:09.880 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:47:09.880 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:47:09.880 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:47:09.880 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:47:09.883 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:47:09.883 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.129 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:47:09.884 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:47:39.745 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:47:39.746 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:47:39.755 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:47:39.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:47:39.793 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:47:39.793 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:47:39.793 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:47:39.793 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:47:39.793 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:47:39.793 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:47:39.793 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:47:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:47:39.796 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:47:39.797 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:47:49.797 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:47:49.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:47:49.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:47:49.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:47:49.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:47:49.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:47:49.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:47:49.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:47:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:47:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:47:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:48:09.834 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:48:09.868 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:48:09.868 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:48:09.868 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:48:09.868 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:48:09.868 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:48:09.868 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:48:09.868 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:48:09.870 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:48:09.871 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.115 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:48:09.871 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:48:39.742 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:48:39.743 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:48:39.750 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:48:39.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:48:39.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:48:39.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:48:39.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:48:39.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:48:39.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:48:39.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:48:39.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:48:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:48:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:48:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:48:49.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:48:49.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:48:49.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:48:49.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:48:49.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:48:49.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:48:49.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:48:49.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:48:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:48:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:48:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:49:09.824 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:49:09.857 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:49:09.857 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:49:09.857 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:49:09.857 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:49:09.857 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:49:09.857 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:49:09.857 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:49:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:49:09.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:49:09.860 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:49:39.742 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:49:39.742 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:49:39.749 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:49:39.749 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:49:39.786 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:49:39.786 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:49:39.786 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:49:39.786 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:49:39.786 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:49:39.786 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:49:39.786 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:49:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:49:39.789 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.039 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:49:39.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:49:49.790 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:49:49.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:49:49.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:49:49.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:49:49.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:49:49.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:49:49.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:49:49.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:49:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:49:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:49:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:50:09.826 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:50:09.880 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:50:09.880 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:50:09.880 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:50:09.880 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:50:09.880 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:50:09.880 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:50:09.880 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:50:09.883 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:50:09.884 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.134 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:50:09.884 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:50:39.745 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:50:39.746 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:50:39.757 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:50:39.758 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:50:39.801 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:50:39.801 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:50:39.801 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:50:39.801 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:50:39.801 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:50:39.801 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:50:39.801 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:50:39.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:50:39.804 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.046 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:50:39.805 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:50:49.805 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:50:49.844 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:50:49.844 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:50:49.844 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:50:49.844 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:50:49.844 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:50:49.844 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:50:49.844 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:50:49.845 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:50:49.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.088 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:50:49.846 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:51:09.847 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:51:09.880 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:51:09.880 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:51:09.880 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:51:09.880 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:51:09.880 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:51:09.880 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:51:09.880 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:51:09.882 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:51:09.882 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.125 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:51:09.883 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:51:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:51:39.745 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:51:39.753 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:51:39.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:51:39.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:51:39.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:51:39.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:51:39.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:51:39.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:51:39.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:51:39.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:51:39.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:51:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:51:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:51:49.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:51:49.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:51:49.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:51:49.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:51:49.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:51:49.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:51:49.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:51:49.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:51:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:51:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:51:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:52:09.826 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:52:09.860 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:52:09.860 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:52:09.860 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:52:09.860 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:52:09.860 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:52:09.860 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:52:09.860 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:52:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:52:09.863 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.109 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:52:09.863 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:52:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:52:39.745 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:52:39.751 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:52:39.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:52:39.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:52:39.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:52:39.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:52:39.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:52:39.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:52:39.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:52:39.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:52:39.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:52:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:52:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:52:49.793 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:52:49.827 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:52:49.827 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:52:49.827 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:52:49.827 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:52:49.827 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:52:49.827 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:52:49.827 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:52:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:52:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:52:49.830 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:53:09.830 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:53:09.867 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:53:09.867 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:53:09.867 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:53:09.867 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:53:09.867 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:53:09.867 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:53:09.867 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:53:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:53:09.870 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:53:09.870 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:53:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:53:39.742 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:53:39.749 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:53:39.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:53:39.783 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:53:39.783 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:53:39.783 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:53:39.783 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:53:39.783 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:53:39.783 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:53:39.783 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:53:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:53:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:53:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:53:49.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:53:49.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:53:49.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:53:49.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:53:49.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:53:49.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:53:49.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:53:49.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:53:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:53:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:53:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:54:09.824 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:54:09.859 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:54:09.859 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:54:09.859 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:54:09.859 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:54:09.859 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:54:09.859 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:54:09.859 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:54:09.861 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:54:09.861 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:54:09.861 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:54:39.743 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:54:39.743 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:54:39.750 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:54:39.751 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:54:39.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:54:39.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:54:39.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:54:39.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:54:39.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:54:39.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:54:39.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:54:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:54:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:54:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:54:49.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:54:49.821 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:54:49.821 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:54:49.821 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:54:49.821 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:54:49.821 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:54:49.821 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:54:49.821 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:54:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:54:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:54:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:55:09.824 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:55:09.863 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:55:09.863 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:55:09.863 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:55:09.863 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:55:09.863 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:55:09.863 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:55:09.863 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:55:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:55:09.870 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:55:09.870 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:55:39.742 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:55:39.743 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:55:39.756 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:55:39.757 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:55:39.797 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:55:39.797 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:55:39.797 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:55:39.797 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:55:39.797 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:55:39.797 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:55:39.797 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:55:39.799 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:55:39.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:55:39.800 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:55:49.800 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:55:49.834 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:55:49.834 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:55:49.834 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:55:49.834 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:55:49.834 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:55:49.834 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:55:49.834 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:55:49.836 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:55:49.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.08 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:55:49.837 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:56:09.837 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:56:09.875 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:56:09.875 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:56:09.875 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:56:09.875 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:56:09.875 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:56:09.875 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:56:09.875 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:56:09.877 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:56:09.878 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:56:09.878 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:56:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:56:39.740 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:56:39.747 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:56:39.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:56:39.781 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:56:39.781 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:56:39.781 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:56:39.781 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:56:39.781 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:56:39.781 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:56:39.781 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:56:39.783 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:56:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:56:39.784 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:56:49.785 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:56:49.820 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:56:49.820 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:56:49.820 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:56:49.820 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:56:49.820 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:56:49.820 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:56:49.820 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:56:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:56:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.074 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:56:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:57:09.823 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:57:09.862 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:57:09.862 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:57:09.862 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:57:09.862 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:57:09.862 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:57:09.862 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:57:09.862 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:57:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:57:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:57:09.866 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:57:39.737 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:57:39.738 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:57:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:57:39.745 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:57:39.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:57:39.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:57:39.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:57:39.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:57:39.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:57:39.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:57:39.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:57:39.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:57:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:57:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:57:49.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:57:49.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:57:49.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:57:49.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:57:49.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:57:49.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:57:49.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:57:49.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:57:49.816 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:57:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:57:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:58:09.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:58:09.853 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:58:09.853 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:58:09.853 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:58:09.853 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:58:09.853 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:58:09.853 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:58:09.853 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:58:09.855 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:58:09.856 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:58:09.857 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:58:39.738 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:58:39.739 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:58:39.746 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:58:39.746 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:58:39.778 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:58:39.778 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:58:39.778 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:58:39.778 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:58:39.778 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:58:39.778 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:58:39.778 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:58:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:58:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.033 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:58:39.780 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:58:49.781 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:58:49.814 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:58:49.814 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:58:49.814 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:58:49.814 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:58:49.814 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:58:49.814 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:58:49.814 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:58:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:58:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:58:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:59:09.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:59:09.849 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:59:09.849 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:59:09.849 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:59:09.849 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:59:09.849 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:59:09.849 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:59:09.849 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:59:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:59:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.105 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 07:59:09.852 139328 ERROR oslo_service.periodic_task 
>2018-06-25 07:59:39.743 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:59:39.744 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 07:59:39.750 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 07:59:39.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:59:39.784 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:59:39.784 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:59:39.784 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:59:39.784 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:59:39.784 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:59:39.784 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:59:39.784 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:59:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:59:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:59:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 07:59:49.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 07:59:49.818 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 07:59:49.818 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 07:59:49.818 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 07:59:49.818 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 07:59:49.818 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 07:59:49.818 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 07:59:49.818 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 07:59:49.820 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 07:59:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 07:59:49.821 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:00:09.822 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:00:09.859 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:00:09.859 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:00:09.859 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:00:09.859 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:00:09.859 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:00:09.859 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:00:09.859 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:00:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:00:09.862 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:00:09.863 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:00:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:00:39.745 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:00:39.755 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:00:39.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:00:39.798 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:00:39.798 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:00:39.798 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:00:39.798 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:00:39.798 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:00:39.798 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:00:39.798 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:00:39.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:00:39.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:00:39.801 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:00:49.802 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:00:49.838 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:00:49.838 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:00:49.838 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:00:49.838 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:00:49.838 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:00:49.838 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:00:49.838 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:00:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:00:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.084 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:00:49.840 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:01:09.841 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:01:09.881 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:01:09.881 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:01:09.881 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:01:09.881 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:01:09.881 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:01:09.881 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:01:09.881 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:01:09.883 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:01:09.884 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.128 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:01:09.884 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:01:39.745 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:01:39.746 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:01:39.754 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:01:39.755 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:01:39.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:01:39.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:01:39.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:01:39.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:01:39.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:01:39.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:01:39.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:01:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:01:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:01:39.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:01:49.792 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:01:49.824 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:01:49.824 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:01:49.824 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:01:49.824 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:01:49.824 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:01:49.824 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:01:49.824 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:01:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:01:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:01:49.826 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:02:09.827 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:02:09.863 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:02:09.863 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:02:09.863 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:02:09.863 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:02:09.863 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:02:09.863 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:02:09.863 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:02:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:02:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.111 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:02:09.866 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:02:39.747 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:02:39.748 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:02:39.755 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:02:39.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:02:39.791 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:02:39.791 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:02:39.791 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:02:39.791 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:02:39.791 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:02:39.791 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:02:39.791 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:02:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:02:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:02:39.794 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:02:49.795 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:02:49.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:02:49.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:02:49.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:02:49.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:02:49.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:02:49.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:02:49.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:02:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:02:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:02:49.829 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:03:09.830 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:03:09.864 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:03:09.864 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:03:09.864 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:03:09.864 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:03:09.864 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:03:09.864 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:03:09.864 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:03:09.866 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:03:09.866 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:03:09.867 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:03:39.748 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:03:39.748 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:03:39.755 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:03:39.756 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:03:39.790 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:03:39.790 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:03:39.790 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:03:39.790 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:03:39.790 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:03:39.790 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:03:39.790 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:03:39.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:03:39.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:03:39.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:03:49.793 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:03:49.831 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:03:49.831 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:03:49.831 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:03:49.831 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:03:49.831 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:03:49.831 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:03:49.831 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:03:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:03:49.833 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:03:49.834 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:04:09.834 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:04:09.870 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:04:09.870 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:04:09.870 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:04:09.870 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:04:09.870 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:04:09.870 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:04:09.870 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:04:09.872 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:04:09.873 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:04:09.873 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:04:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:04:39.745 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:04:39.752 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:04:39.753 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:04:39.788 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:04:39.788 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:04:39.788 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:04:39.788 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:04:39.788 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:04:39.788 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:04:39.788 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:04:39.790 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:04:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.038 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:04:39.791 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:04:49.791 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:04:49.825 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:04:49.825 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:04:49.825 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:04:49.825 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:04:49.825 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:04:49.825 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:04:49.825 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:04:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:04:49.827 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:04:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:05:09.828 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:05:09.861 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:05:09.861 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:05:09.861 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:05:09.861 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:05:09.861 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:05:09.861 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:05:09.861 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:05:09.863 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:05:09.863 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:05:09.863 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:05:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:05:39.745 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:05:39.751 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:05:39.752 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:05:39.785 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:05:39.785 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:05:39.785 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:05:39.785 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:05:39.785 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:05:39.785 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:05:39.785 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:05:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:05:39.787 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:05:39.788 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:05:49.788 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:05:49.822 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:05:49.822 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:05:49.822 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:05:49.822 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:05:49.822 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:05:49.822 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:05:49.822 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:05:49.823 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:05:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.072 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:05:49.824 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:06:09.824 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:06:09.856 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:06:09.856 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:06:09.856 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:06:09.856 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:06:09.856 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:06:09.856 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:06:09.856 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:06:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:06:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.108 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:06:09.860 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:06:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:06:39.742 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:06:39.750 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:06:39.750 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:06:39.789 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:06:39.789 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:06:39.789 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:06:39.789 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:06:39.789 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:06:39.789 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:06:39.789 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:06:39.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:06:39.792 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:06:39.793 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:06:49.793 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:06:49.826 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:06:49.826 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:06:49.826 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:06:49.826 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:06:49.826 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:06:49.826 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:06:49.826 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:06:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:06:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:06:49.828 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:07:09.829 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:07:09.862 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:07:09.862 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:07:09.862 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:07:09.862 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:07:09.862 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:07:09.862 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:07:09.862 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:07:09.864 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:07:09.865 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:07:09.865 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:07:39.736 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:07:39.737 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:07:39.744 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:07:39.744 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:07:39.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:07:39.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:07:39.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:07:39.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:07:39.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:07:39.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:07:39.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:07:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:07:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.032 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:07:39.778 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:07:49.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:07:49.812 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:07:49.812 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:07:49.812 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:07:49.812 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:07:49.812 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:07:49.812 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:07:49.812 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:07:49.814 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:07:49.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.07 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:07:49.815 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:08:09.815 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:08:09.848 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:08:09.848 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:08:09.848 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:08:09.848 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:08:09.848 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:08:09.848 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:08:09.848 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:08:09.850 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:08:09.851 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.106 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:08:09.851 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:08:39.732 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:08:39.733 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:08:39.741 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:08:39.742 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:08:39.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:08:39.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:08:39.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:08:39.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:08:39.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:08:39.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:08:39.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:08:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:08:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:08:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:08:49.787 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:08:49.823 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:08:49.823 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:08:49.823 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:08:49.823 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:08:49.823 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:08:49.823 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:08:49.823 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:08:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:08:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.082 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:08:49.825 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:09:09.826 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:09:09.867 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:09:09.867 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:09:09.867 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:09:09.867 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:09:09.867 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:09:09.867 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:09:09.867 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:09:09.869 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:09:09.870 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.127 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:09:09.870 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:09:39.731 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:09:39.732 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:09:39.740 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:09:39.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:09:39.779 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:09:39.779 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:09:39.779 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:09:39.779 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:09:39.779 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:09:39.779 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:09:39.779 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:09:39.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:09:39.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:09:39.781 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:09:49.782 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:09:49.815 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:09:49.815 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:09:49.815 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:09:49.815 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:09:49.815 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:09:49.815 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:09:49.815 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:09:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:09:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.076 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:09:49.817 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:10:09.818 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:10:09.856 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:10:09.856 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:10:09.856 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:10:09.856 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:10:09.856 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:10:09.856 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:10:09.856 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:10:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:10:09.859 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.118 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:10:09.859 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:10:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:10:39.731 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:10:39.738 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:10:39.739 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:10:39.772 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:10:39.772 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:10:39.772 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:10:39.772 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:10:39.772 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:10:39.772 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:10:39.772 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:10:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:10:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.036 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:10:39.775 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:10:49.776 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:10:49.810 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:10:49.810 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:10:49.810 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:10:49.810 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:10:49.810 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:10:49.810 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:10:49.810 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:10:49.811 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:10:49.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:10:49.812 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:11:09.813 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:11:09.847 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:11:09.847 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:11:09.847 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:11:09.847 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:11:09.847 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:11:09.847 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:11:09.847 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:11:09.848 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:11:09.849 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.11 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:11:09.849 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:11:39.730 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:11:39.731 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:11:39.739 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:11:39.740 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:11:39.775 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:11:39.775 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:11:39.775 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:11:39.775 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:11:39.775 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:11:39.775 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:11:39.775 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:11:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:11:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.037 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:11:39.777 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:11:49.778 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:11:49.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:11:49.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:11:49.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:11:49.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:11:49.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:11:49.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:11:49.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:11:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:11:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:11:49.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:12:09.819 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:12:09.851 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:12:09.851 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:12:09.851 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:12:09.851 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:12:09.851 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:12:09.851 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:12:09.851 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:12:09.853 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:12:09.854 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:12:09.854 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:12:39.735 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:12:39.736 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:12:39.743 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:12:39.743 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:12:39.782 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:12:39.782 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:12:39.782 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:12:39.782 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:12:39.782 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:12:39.782 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:12:39.782 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:12:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:12:39.785 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.041 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:12:39.786 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:12:49.786 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:12:49.819 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:12:49.819 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:12:49.819 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:12:49.819 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:12:49.819 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:12:49.819 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:12:49.819 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:12:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:12:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.078 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:12:49.822 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:13:09.823 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:13:09.856 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:13:09.856 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:13:09.856 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:13:09.856 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:13:09.856 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:13:09.856 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:13:09.856 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:13:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:13:09.858 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.114 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:13:09.858 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:13:39.740 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:13:39.740 139328 DEBUG cinder.manager [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /usr/lib/python2.7/site-packages/cinder/manager.py:189
>2018-06-25 08:13:39.747 139328 DEBUG oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:13:39.747 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:13:39.780 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:13:39.780 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:13:39.780 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:13:39.780 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:13:39.780 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:13:39.780 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:13:39.780 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:13:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:13:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 0.034 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:13:39.782 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:13:49.783 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:13:49.816 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:13:49.816 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:13:49.816 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:13:49.816 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:13:49.816 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:13:49.816 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:13:49.816 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:13:49.818 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:13:49.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 10.071 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:13:49.819 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:14:09.819 139328 DEBUG cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:14:09.858 139328 ERROR cinder.volume.drivers.rbd [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:14:09.858 139328 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:14:09.858 139328 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:14:09.858 139328 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:14:09.858 139328 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:14:09.858 139328 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:14:09.858 139328 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:14:09.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:14:09.860 139328 DEBUG cinder.utils [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task [req-e679111c-4839-434c-8828-1b058c0edb30 - - - - -] Error during VolumeManager._report_driver_status: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task Traceback (most recent call last):
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     task(self, context)
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2367, in _report_driver_status
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     volume_stats = self.driver.get_volume_stats(refresh=True)
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 483, in get_volume_stats
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     self._update_volume_stats()
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 466, in _update_volume_stats
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     free_capacity, total_capacity = self._get_pool_stats()
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 403, in _get_pool_stats
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     with RADOSClient(self) as client:
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     return _do_conn(pool, remote, timeout)
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     return r.call(f, *args, **kwargs)
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     raise attempt.get()
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:14:09.861 139328 ERROR oslo_service.periodic_task 
>2018-06-25 08:14:25.967 139116 INFO oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Caught SIGTERM, stopping children
>2018-06-25 08:14:25.971 139116 DEBUG oslo_concurrency.lockutils [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 08:14:25.971 139116 DEBUG oslo_concurrency.lockutils [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 08:14:25.972 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Stop services. stop /usr/lib/python2.7/site-packages/oslo_service/service.py:624
>2018-06-25 08:14:25.972 139116 DEBUG oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Killing children. stop /usr/lib/python2.7/site-packages/oslo_service/service.py:629
>2018-06-25 08:14:25.973 139116 INFO oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Waiting on 1 children to exit
>2018-06-25 08:14:27.812 139328 DEBUG oslo_concurrency.lockutils [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 08:14:27.812 139328 DEBUG oslo_concurrency.lockutils [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 08:14:39.799 139116 INFO oslo_service.service [req-563ade2f-0efd-41b2-a14f-d9dd6969b919 - - - - -] Child 139328 exited with status 0
>2018-06-25 08:14:42.560 938083 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
>2018-06-25 08:14:42.560 938083 DEBUG oslo_concurrency.lockutils [-] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 08:14:42.561 938083 DEBUG oslo_concurrency.lockutils [-] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 08:14:42.966 938083 DEBUG oslo_db.sqlalchemy.engines [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:14:43.113 938083 INFO cinder.rpc [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version.
>2018-06-25 08:14:43.118 938083 INFO cinder.rpc [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version.
>2018-06-25 08:14:43.123 938083 DEBUG oslo_db.sqlalchemy.engines [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:14:43.131 938083 WARNING cinder.keymgr.conf_key_mgr [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] This key manager is insecure and is not recommended for production deployments
>2018-06-25 08:14:43.142 938083 INFO cinder.volume.manager [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Image-volume cache disabled for host hostgroup@tripleo_ceph.
>2018-06-25 08:14:43.148 938083 DEBUG oslo_db.sqlalchemy.engines [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:14:43.161 938083 INFO oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Starting 1 workers
>2018-06-25 08:14:43.164 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Started child 938334 _start_child /usr/lib/python2.7/site-packages/oslo_service/service.py:513
>2018-06-25 08:14:43.167 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Full set of CONF: wait /usr/lib/python2.7/site-packages/oslo_service/service.py:580
>2018-06-25 08:14:43.168 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2865
>2018-06-25 08:14:43.168 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Configuration options gathered from: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2866
>2018-06-25 08:14:43.168 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] command line args: ['--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--logfile', '/var/log/cinder/volume.log'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2867
>2018-06-25 08:14:43.169 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] config files: ['/usr/share/cinder/cinder-dist.conf', '/etc/cinder/cinder.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2869
>2018-06-25 08:14:43.169 938334 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0)
>2018-06-25 08:14:43.169 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] ================================================================================ log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2870
>2018-06-25 08:14:43.170 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] allow_availability_zone_fallback = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.171 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] allowed_direct_url_schemes     = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.171 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] api_rate_limit                 = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.172 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] auth_strategy                  = keystone log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.172 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_availability_zone      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.172 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_host                   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.173 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backup_api_class               = cinder.backup.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.173 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backup_manager                 = cinder.backup.manager.BackupManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.173 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backup_name_template           = backup-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.174 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backup_use_same_host           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.174 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backup_use_temp_snapshot       = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.175 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] chap_password                  = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.175 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] chap_username                  =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.175 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.176 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] cinder_internal_tenant_project_id = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.176 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] cinder_internal_tenant_user_id = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.176 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] cluster                        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.177 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] compute_api_class              = cinder.compute.nova.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.177 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] config_dir                     = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.177 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] config_file                    = ['/usr/share/cinder/cinder-dist.conf', '/etc/cinder/cinder.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.178 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] consistencygroup_api_class     = cinder.consistencygroup.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.178 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] control_exchange               = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.178 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] db_driver                      = cinder.db log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.179 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] debug                          = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.179 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] default_availability_zone      = nova log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.179 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] default_group_type             = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.180 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'dogpile.core.dogpile=INFO'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.180 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] default_volume_type            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.181 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] driver_client_cert             = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.181 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] driver_client_cert_key         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.182 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] driver_data_namespace          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.182 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] driver_ssl_cert_path           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.183 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] driver_ssl_cert_verify         = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.183 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] driver_use_ssl                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.183 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] enable_new_services            = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.184 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] enable_unsupported_driver      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.184 938334 INFO cinder.volume.manager [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Starting volume driver RBDDriver (1.2.0)
>2018-06-25 08:14:43.185 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] enable_v1_api                  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.185 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] enable_v2_api                  = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.185 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] enable_v3_api                  = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.185 938334 DEBUG cinder.volume.drivers.rbd [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:14:43.186 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] enabled_backends               = ['tripleo_ceph'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.186 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] enforce_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.186 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] fatal_exception_format_errors  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.187 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] filter_function                = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.187 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] glance_api_insecure            = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.187 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] glance_api_servers             = ['http://172.17.1.12:9292'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.188 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] glance_api_ssl_compression     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.188 938083 WARNING oslo_config.cfg [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens).  Its value may be silently ignored in the future.
>2018-06-25 08:14:43.189 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] glance_api_version             = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.189 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] glance_ca_certificates_file    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.189 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] glance_catalog_info            = image:glance:publicURL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.190 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.190 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] glance_num_retries             = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.191 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] glance_request_timeout         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.191 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] goodness_function              = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.192 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] graceful_shutdown_timeout      = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.192 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] group_api_class                = cinder.group.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.193 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] host                           = hostgroup log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.193 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iet_conf                       = /etc/iet/ietd.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.194 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] image_conversion_dir           = /var/lib/cinder/conversion log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.194 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] image_upload_use_cinder_backend = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.194 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] image_upload_use_internal_tenant = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.195 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] image_volume_cache_enabled     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.195 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] image_volume_cache_max_count   = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.195 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] image_volume_cache_max_size_gb = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.196 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] instance_format                = [instance: %(uuid)s]  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.196 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] instance_uuid_format           = [instance: %(uuid)s]  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.196 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iscsi_helper                   = lioadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.197 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iscsi_iotype                   = fileio log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.197 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iscsi_ip_address               = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.198 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iscsi_port                     = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.198 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iscsi_protocol                 = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.198 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iscsi_secondary_ip_addresses   = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.199 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iscsi_target_flags             =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.199 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iscsi_target_prefix            = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.200 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iscsi_write_cache              = on log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.200 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iser_helper                    = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.201 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iser_ip_address                = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.201 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iser_port                      = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.201 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] iser_target_prefix             = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.202 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] log_config_append              = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.202 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] log_date_format                = %Y-%m-%d %H:%M:%S log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.202 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] log_dir                        = /var/log/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.203 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] log_file                       = /var/log/cinder/volume.log log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.203 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] log_options                    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.203 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] logging_context_format_string  = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.204 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] logging_debug_format_suffix    = %(funcName)s %(pathname)s:%(lineno)d log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.204 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] logging_default_format_string  = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.204 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] logging_exception_prefix       = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.205 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] logging_user_identity_format   = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.205 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] max_age                        = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.205 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] max_over_subscription_ratio    = 20.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.206 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] message_reap_interval          = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.206 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] message_ttl                    = 2592000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.206 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] migration_create_volume_timeout_secs = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.207 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] monkey_patch                   = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.207 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] monkey_patch_modules           = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.207 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] my_ip                          = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.208 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] no_snapshot_gb_quota           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.208 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] num_iser_scan_tries            = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.208 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] num_shell_tries                = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.209 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] num_volume_device_scan_tries   = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.209 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] os_privileged_user_auth_url    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.209 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] os_privileged_user_name        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.210 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] os_privileged_user_password    = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.210 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] os_privileged_user_tenant      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.210 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] osapi_max_limit                = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.211 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] osapi_volume_base_URL          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.211 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] osapi_volume_ext_list          = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.211 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.212 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] osapi_volume_listen            = controller-0.internalapi.localdomain log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.212 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] osapi_volume_listen_port       = 8776 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.213 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] osapi_volume_use_ssl           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.213 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] osapi_volume_workers           = 4 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.213 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] per_volume_size_limit          = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.214 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] periodic_fuzzy_delay           = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.214 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] periodic_interval              = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.214 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] public_endpoint                = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.214 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] publish_errors                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.215 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] query_volume_filters           = ['name', 'status', 'metadata', 'availability_zone', 'bootable', 'group_id'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.215 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] quota_backup_gigabytes         = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.216 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] quota_backups                  = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.216 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] quota_consistencygroups        = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.216 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] quota_driver                   = cinder.quota.DbQuotaDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.217 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] quota_gigabytes                = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.217 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] quota_groups                   = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.217 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] quota_snapshots                = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.218 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] quota_volumes                  = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.218 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] rate_limit_burst               = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.218 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] rate_limit_except_level        = CRITICAL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.219 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] rate_limit_interval            = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.219 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] replication_device             = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.220 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] report_discard_supported       = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.220 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] report_interval                = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.220 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] reservation_clean_interval     = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.221 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] reservation_expire             = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.221 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] reserved_percentage            = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.221 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] resource_query_filters_file    = /etc/cinder/resource_filters.json log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.222 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] rootwrap_config                = /etc/cinder/rootwrap.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.222 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] rpc_backend                    = rabbit log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.222 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] rpc_response_timeout           = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.223 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] run_external_periodic_tasks    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.223 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] scheduler_manager              = cinder.scheduler.manager.SchedulerManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.223 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] scst_target_driver             = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.224 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] scst_target_iqn_name           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.224 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] service_down_time              = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.224 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] snapshot_name_template         = snapshot-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.225 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] state_path                     = /var/lib/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.225 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] storage_availability_zone      = nova log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.226 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] storage_protocol               = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.226 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] syslog_log_facility            = LOG_USER log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.226 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] trace_flags                    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.227 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] transfer_api_class             = cinder.transfer.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.227 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] transport_url                  = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.228 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] until_refresh                  = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.228 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] use_chap_auth                  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.228 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] use_default_quota_class        = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.229 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] use_journal                    = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.227 938334 ERROR cinder.volume.drivers.rbd [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:14:43.227 938334 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:14:43.227 938334 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:14:43.227 938334 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:14:43.227 938334 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:14:43.227 938334 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:14:43.227 938334 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:14:43.229 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] use_multipath_for_image_xfer   = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.229 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] use_stderr                     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.230 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] use_syslog                     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.230 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_api_class               = cinder.volume.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.230 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_backend_name            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.231 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_clear                   = zero log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.231 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_clear_ionice            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.231 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_clear_size              = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.232 938334 DEBUG cinder.utils [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:14:43.232 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_copy_blkio_cgroup_name  = cinder-volume-copy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.232 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_copy_bps_limit          = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.232 938334 DEBUG cinder.utils [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Have been at this for 0.046 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:14:43.233 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_dd_blocksize            = 1M log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.233 938334 DEBUG cinder.utils [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:14:43.233 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_manager                 = cinder.volume.manager.VolumeManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.233 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_name_template           = volume-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.234 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_service_inithost_offload = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.234 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volume_usage_audit_period      = month log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.234 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] volumes_dir                    = /etc/cinder/volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.235 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] watch_log_file                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.235 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] zoning_mode                    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:14:43.236 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.amqp_auto_delete = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.236 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.amqp_durable_queues = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.236 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.conn_pool_min_size = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.237 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.conn_pool_ttl = 1200 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.237 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.fake_rabbit = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.237 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.heartbeat_rate = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.238 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.238 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.kombu_compression = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.239 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.kombu_failover_strategy = round-robin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.239 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.239 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.240 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_ha_queues = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.240 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_host = localhost log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.241 938083 WARNING oslo_config.cfg [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:14:43.241 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_hosts = ['localhost:5672'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.241 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_interval_max = 30 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.242 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.242 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_max_retries = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.242 938083 WARNING oslo_config.cfg [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:14:43.243 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.243 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_port = 5672 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.244 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.244 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_retry_backoff = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.244 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_retry_interval = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.245 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.245 938083 WARNING oslo_config.cfg [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:14:43.245 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_userid = guest log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.246 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rabbit_virtual_host = / log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.246 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.rpc_conn_pool_size = 30 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.246 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.ssl      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.247 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.ssl_ca_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.247 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.ssl_cert_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.247 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.ssl_key_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.248 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_rabbit.ssl_version =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.248 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] coordination.backend_url       = file:///var/lib/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.249 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] coordination.heartbeat         = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.249 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] coordination.initial_reconnect_backoff = 0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.249 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] coordination.max_reconnect_backoff = 60.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.250 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.backend               = sqlalchemy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.250 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.connection            = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.251 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.connection_debug      = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.251 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.connection_trace      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.251 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.db_inc_retry_interval = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.252 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.db_max_retries        = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.252 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.db_max_retry_interval = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.253 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.db_retry_interval     = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.253 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.idle_timeout          = 3600 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.253 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.max_overflow          = 50 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.254 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.max_pool_size         = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.254 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.max_retries           = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.254 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.min_pool_size         = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.255 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.mysql_enable_ndb      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.255 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.mysql_sql_mode        = TRADITIONAL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.256 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.pool_timeout          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.256 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.retry_interval        = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.256 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.slave_connection      = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.257 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.sqlite_synchronous    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.257 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] database.use_db_reconnect      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.257 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_notifications.driver = ['messagingv2'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.258 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_notifications.retry = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.258 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_notifications.topics = ['notifications'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.258 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_messaging_notifications.transport_url = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.259 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_reports.file_event_handler = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.259 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_reports.file_event_handler_interval = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.260 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_reports.log_dir           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.260 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.backend_availability_zone = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.260 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.backend_host      = hostgroup log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.261 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.backup_use_temp_snapshot = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.261 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.chap_password     = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.261 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.chap_username     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.262 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.chiscsi_conf      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.262 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.driver_client_cert = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.263 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.driver_client_cert_key = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.263 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.driver_data_namespace = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.263 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.driver_ssl_cert_path = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.264 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.driver_ssl_cert_verify = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.264 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.driver_use_ssl    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.264 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.enable_unsupported_driver = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.265 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.enforce_multipath_for_image_xfer = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.265 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.extra_capabilities = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.266 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.filter_function   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.266 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.goodness_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.266 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iet_conf          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.267 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.image_upload_use_cinder_backend = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.267 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.image_upload_use_internal_tenant = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.267 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.image_volume_cache_enabled = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.268 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.image_volume_cache_max_count = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.268 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.image_volume_cache_max_size_gb = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.268 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iscsi_helper      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.269 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iscsi_iotype      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.269 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iscsi_ip_address  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.269 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iscsi_port        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.270 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iscsi_protocol    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.270 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iscsi_secondary_ip_addresses = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.270 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iscsi_target_flags = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.271 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iscsi_target_prefix = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.271 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iscsi_write_cache = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.271 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iser_helper       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.272 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iser_ip_address   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.272 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iser_port         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.272 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.iser_target_prefix = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.273 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.max_over_subscription_ratio = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.273 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.num_iser_scan_tries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.273 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.num_shell_tries   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.274 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.num_volume_device_scan_tries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.274 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rados_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.274 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rados_connection_interval = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.275 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rados_connection_retries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.275 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rbd_ceph_conf     = /etc/ceph/ceph.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.275 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rbd_cluster_name  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.276 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rbd_flatten_volume_from_snapshot = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.276 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rbd_keyring_conf  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.276 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rbd_max_clone_depth = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.277 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rbd_pool          = volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.277 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rbd_secret_uuid   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.277 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rbd_store_chunk_size = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.278 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.rbd_user          = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.278 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.replication_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.278 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.replication_device = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.279 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.report_discard_supported = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.279 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.report_dynamic_total_capacity = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.279 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.reserved_percentage = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.280 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.scst_target_driver = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.280 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.scst_target_iqn_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.280 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.storage_protocol  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.281 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.suppress_requests_ssl_warnings = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.281 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.trace_flags       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.281 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.use_chap_auth     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.282 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.use_multipath_for_image_xfer = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.282 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.volume_backend_name = tripleo_ceph log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.282 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.volume_clear      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.283 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.volume_clear_ionice = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.283 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.volume_clear_size = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.283 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.volume_copy_blkio_cgroup_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.284 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.volume_copy_bps_limit = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.284 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.volume_dd_blocksize = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.284 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.volume_driver     = cinder.volume.drivers.rbd.RBDDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.285 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] tripleo_ceph.volumes_dir       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.285 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] profiler.connection_string     = messaging:// log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.285 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] profiler.enabled               = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.286 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] profiler.es_doc_type           = notification log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.286 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] profiler.es_scroll_size        = 10000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.286 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] profiler.es_scroll_time        = 2m log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.287 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] profiler.hmac_keys             = SECRET_KEY log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.287 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] profiler.sentinel_service_name = mymaster log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.287 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] profiler.socket_timeout        = 0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.288 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] profiler.trace_sqlalchemy      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.288 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] privsep_osbrick.capabilities   = [21] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.289 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] privsep_osbrick.group          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.289 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] privsep_osbrick.helper_command = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.289 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] privsep_osbrick.user           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.290 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.backend_availability_zone = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.290 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.backup_use_temp_snapshot = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.290 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.chap_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.291 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.chap_username =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.291 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.291 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.driver_client_cert = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.292 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.driver_client_cert_key = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.292 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.driver_data_namespace = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.292 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.driver_ssl_cert_path = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.293 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.driver_ssl_cert_verify = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.293 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.driver_use_ssl = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.293 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.enable_unsupported_driver = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.294 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.enforce_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.294 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.extra_capabilities = {} log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.294 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.filter_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.295 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.goodness_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.295 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iet_conf      = /etc/iet/ietd.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.295 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.image_upload_use_cinder_backend = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.296 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.image_upload_use_internal_tenant = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.296 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.image_volume_cache_enabled = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.296 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.image_volume_cache_max_count = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.297 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.image_volume_cache_max_size_gb = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.297 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iscsi_helper  = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.297 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iscsi_iotype  = fileio log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.298 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iscsi_ip_address = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.298 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iscsi_port    = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.298 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iscsi_protocol = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.299 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iscsi_secondary_ip_addresses = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.299 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iscsi_target_flags =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.300 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iscsi_target_prefix = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.300 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iscsi_write_cache = on log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.300 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iser_helper   = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.301 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iser_ip_address = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.301 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iser_port     = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.302 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.302 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.max_over_subscription_ratio = 20.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.302 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.num_iser_scan_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.302 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.num_shell_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.303 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.num_volume_device_scan_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.303 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rados_connect_timeout = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.304 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rados_connection_interval = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.304 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rados_connection_retries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.304 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rbd_ceph_conf =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.305 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rbd_cluster_name = ceph log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.305 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rbd_flatten_volume_from_snapshot = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.305 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rbd_keyring_conf =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.306 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rbd_max_clone_depth = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.306 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rbd_pool      = rbd log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.306 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rbd_secret_uuid = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.307 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rbd_store_chunk_size = 4 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.307 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.rbd_user      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.307 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.replication_connect_timeout = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.308 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.replication_device = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.308 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.report_discard_supported = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.309 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.report_dynamic_total_capacity = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.309 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.reserved_percentage = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.309 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.scst_target_driver = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.310 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.scst_target_iqn_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.310 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.storage_protocol = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.310 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.suppress_requests_ssl_warnings = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.311 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.trace_flags   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.311 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.use_chap_auth = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.311 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.use_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.312 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.volume_backend_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.312 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.volume_clear  = zero log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.312 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.volume_clear_ionice = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.313 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.volume_clear_size = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.313 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.313 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.volume_copy_bps_limit = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.314 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.volume_dd_blocksize = 1M log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.314 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.315 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] backend_defaults.volumes_dir   = /var/lib/cinder/volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.315 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_concurrency.disable_process_locking = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.316 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_concurrency.lock_path     = /var/lib/cinder/tmp log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.316 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] barbican.auth_endpoint         = http://localhost/identity/v3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.316 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] barbican.barbican_api_version  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.317 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] barbican.barbican_endpoint     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.317 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] barbican.number_of_retries     = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.317 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] barbican.retry_delay           = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.318 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] barbican.verify_ssl            = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.318 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.admin_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.318 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.admin_tenant_name = admin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.319 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.admin_token = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.319 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.admin_user  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.319 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.auth_admin_prefix =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.320 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.auth_host   = 127.0.0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.320 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.auth_port   = 35357 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.321 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.auth_protocol = http log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.321 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.auth_section = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.321 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.auth_type   = password log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.322 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.auth_uri    = http://172.17.1.12:5000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.322 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.auth_version = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.322 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.cache       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.323 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.cafile      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.323 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.certfile    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.324 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.check_revocations_for_cached = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.324 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.delay_auth_decision = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.324 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.enforce_token_bind = permissive log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.325 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.hash_algorithms = ['md5'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.325 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.http_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.325 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.http_request_max_retries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.326 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.identity_uri = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.326 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.include_service_catalog = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.326 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.insecure    = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.327 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.keyfile     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.327 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.memcache_pool_conn_get_timeout = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.327 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.memcache_pool_dead_retry = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.328 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.memcache_pool_maxsize = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.328 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.memcache_pool_socket_timeout = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.328 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.memcache_pool_unused_timeout = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.329 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.memcache_secret_key = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.329 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.memcache_security_strategy = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.329 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.memcache_use_advanced_pool = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.330 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.memcached_servers = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.330 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.region_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.330 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.revocation_cache_time = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.331 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.service_token_roles = ['service'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.331 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.service_token_roles_required = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.331 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.signing_dir = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.332 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] keystone_authtoken.token_cache_time = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.332 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_versionedobjects.fatal_exception_format_errors = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.333 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_policy.policy_default_rule = default log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.333 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_policy.policy_dirs        = ['policy.d'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.333 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] oslo_policy.policy_file        = /etc/cinder/policy.json log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.334 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] key_manager.api_class          = cinder.keymgr.conf_key_mgr.ConfKeyManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.334 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] key_manager.fixed_key          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:14:43.334 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2889
>2018-06-25 08:14:53.233 938334 DEBUG cinder.volume.drivers.rbd [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:14:53.269 938334 ERROR cinder.volume.drivers.rbd [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:14:53.269 938334 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:14:53.269 938334 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:14:53.269 938334 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:14:53.269 938334 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:14:53.269 938334 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:14:53.269 938334 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:14:53.271 938334 DEBUG cinder.utils [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:14:53.271 938334 DEBUG cinder.utils [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Have been at this for 10.085 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:14:53.271 938334 DEBUG cinder.utils [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:15:13.272 938334 DEBUG cinder.volume.drivers.rbd [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:15:13.310 938334 ERROR cinder.volume.drivers.rbd [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:15:13.310 938334 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:15:13.310 938334 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:15:13.310 938334 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:15:13.310 938334 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:15:13.310 938334 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:15:13.310 938334 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:15:13.313 938334 DEBUG cinder.utils [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:15:13.313 938334 DEBUG cinder.utils [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Have been at this for 30.127 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Failed to initialize driver.: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 431, in init_host
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager     self.driver.check_for_setup_error()
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 272, in check_for_setup_error
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager     with RADOSClient(self):
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager     return _do_conn(pool, remote, timeout)
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager     return r.call(f, *args, **kwargs)
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager     raise attempt.get()
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:15:13.314 938334 ERROR cinder.volume.manager 
>2018-06-25 08:15:13.317 938334 DEBUG cinder.service [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Creating RPC server for service cinder-volume start /usr/lib/python2.7/site-packages/cinder/service.py:219
>2018-06-25 08:15:13.325 938334 DEBUG oslo_db.sqlalchemy.engines [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:15:13.332 938334 DEBUG cinder.service [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Pinning object versions for RPC server serializer to 1.27 start /usr/lib/python2.7/site-packages/cinder/service.py:226
>2018-06-25 08:15:13.335 938334 WARNING oslo_config.cfg [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:15:13.335 938334 WARNING oslo_config.cfg [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:15:13.336 938334 WARNING oslo_config.cfg [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:15:13.471 938334 INFO cinder.volume.manager [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Initializing RPC dependent components of volume driver RBDDriver (1.2.0)
>2018-06-25 08:15:13.472 938334 ERROR cinder.utils [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Volume driver RBDDriver not initialized
>2018-06-25 08:15:13.472 938334 ERROR cinder.volume.manager [req-3da94d01-faf7-476b-af88-51ef2ffaf006 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready.
>2018-06-25 08:15:23.473 938334 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:15:33.474 938334 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:15:43.476 938334 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:15:53.477 938334 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:16:03.474 938334 DEBUG oslo_service.periodic_task [req-31c519d1-2b23-4cf9-b8d4-75d903c7046d - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:16:03.475 938334 DEBUG oslo_service.periodic_task [req-31c519d1-2b23-4cf9-b8d4-75d903c7046d - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:16:03.475 938334 WARNING cinder.volume.manager [req-31c519d1-2b23-4cf9-b8d4-75d903c7046d - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:16:03.477 938334 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:16:12.331 938083 INFO oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Caught SIGTERM, stopping children
>2018-06-25 08:16:12.332 938083 DEBUG oslo_concurrency.lockutils [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 08:16:12.332 938083 DEBUG oslo_concurrency.lockutils [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 08:16:12.332 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Stop services. stop /usr/lib/python2.7/site-packages/oslo_service/service.py:624
>2018-06-25 08:16:12.333 938083 DEBUG oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Killing children. stop /usr/lib/python2.7/site-packages/oslo_service/service.py:629
>2018-06-25 08:16:12.333 938083 INFO oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Waiting on 1 children to exit
>2018-06-25 08:16:13.450 938334 DEBUG oslo_concurrency.lockutils [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 08:16:13.451 938334 DEBUG oslo_concurrency.lockutils [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 08:17:03.511 938083 INFO oslo_service.service [req-5928976e-2794-4706-a6be-db1af645a1c9 - - - - -] Child 938334 exited with status 0
>2018-06-25 08:17:07.138 952691 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
>2018-06-25 08:17:07.139 952691 DEBUG oslo_concurrency.lockutils [-] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 08:17:07.139 952691 DEBUG oslo_concurrency.lockutils [-] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 08:17:07.501 952691 DEBUG oslo_db.sqlalchemy.engines [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:17:07.649 952691 INFO cinder.rpc [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version.
>2018-06-25 08:17:07.653 952691 INFO cinder.rpc [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version.
>2018-06-25 08:17:07.659 952691 DEBUG oslo_db.sqlalchemy.engines [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:17:07.667 952691 WARNING cinder.keymgr.conf_key_mgr [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] This key manager is insecure and is not recommended for production deployments
>2018-06-25 08:17:07.676 952691 INFO cinder.volume.manager [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Image-volume cache disabled for host hostgroup@tripleo_ceph.
>2018-06-25 08:17:07.681 952691 DEBUG oslo_db.sqlalchemy.engines [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:17:07.692 952691 INFO oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Starting 1 workers
>2018-06-25 08:17:07.695 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Started child 953284 _start_child /usr/lib/python2.7/site-packages/oslo_service/service.py:513
>2018-06-25 08:17:07.698 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Full set of CONF: wait /usr/lib/python2.7/site-packages/oslo_service/service.py:580
>2018-06-25 08:17:07.699 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2865
>2018-06-25 08:17:07.699 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Configuration options gathered from: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2866
>2018-06-25 08:17:07.699 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] command line args: ['--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--logfile', '/var/log/cinder/volume.log'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2867
>2018-06-25 08:17:07.700 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] config files: ['/usr/share/cinder/cinder-dist.conf', '/etc/cinder/cinder.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2869
>2018-06-25 08:17:07.699 953284 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0)
>2018-06-25 08:17:07.700 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] ================================================================================ log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2870
>2018-06-25 08:17:07.701 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] allow_availability_zone_fallback = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.701 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] allowed_direct_url_schemes     = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.701 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] api_rate_limit                 = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.702 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] auth_strategy                  = keystone log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.702 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_availability_zone      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.703 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_host                   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.703 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backup_api_class               = cinder.backup.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.703 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backup_manager                 = cinder.backup.manager.BackupManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.704 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backup_name_template           = backup-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.704 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backup_use_same_host           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.704 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backup_use_temp_snapshot       = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.705 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] chap_password                  = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.705 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] chap_username                  =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.705 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.706 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] cinder_internal_tenant_project_id = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.706 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] cinder_internal_tenant_user_id = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.706 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] cluster                        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.707 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] compute_api_class              = cinder.compute.nova.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.707 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] config_dir                     = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.707 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] config_file                    = ['/usr/share/cinder/cinder-dist.conf', '/etc/cinder/cinder.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.708 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] consistencygroup_api_class     = cinder.consistencygroup.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.708 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] control_exchange               = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.708 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] db_driver                      = cinder.db log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.708 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] debug                          = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.709 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] default_availability_zone      = nova log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.709 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] default_group_type             = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.710 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'dogpile.core.dogpile=INFO'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.710 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] default_volume_type            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.710 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] driver_client_cert             = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.711 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] driver_client_cert_key         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.711 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] driver_data_namespace          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.711 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] driver_ssl_cert_path           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.712 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] driver_ssl_cert_verify         = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.712 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] driver_use_ssl                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.712 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] enable_new_services            = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.713 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] enable_unsupported_driver      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.713 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] enable_v1_api                  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.713 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] enable_v2_api                  = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.714 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] enable_v3_api                  = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.714 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] enabled_backends               = ['tripleo_ceph'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.714 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] enforce_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.715 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] fatal_exception_format_errors  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.715 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] filter_function                = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.715 953284 INFO cinder.volume.manager [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Starting volume driver RBDDriver (1.2.0)
>2018-06-25 08:17:07.715 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] glance_api_insecure            = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.716 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] glance_api_servers             = ['http://172.17.1.12:9292'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.716 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] glance_api_ssl_compression     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.716 952691 WARNING oslo_config.cfg [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens).  Its value may be silently ignored in the future.
>2018-06-25 08:17:07.716 953284 DEBUG cinder.volume.drivers.rbd [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:17:07.717 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] glance_api_version             = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.717 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] glance_ca_certificates_file    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.717 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] glance_catalog_info            = image:glance:publicURL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.718 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.718 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] glance_num_retries             = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.719 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] glance_request_timeout         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.719 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] goodness_function              = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.719 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] graceful_shutdown_timeout      = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.720 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] group_api_class                = cinder.group.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.721 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] host                           = hostgroup log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.721 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iet_conf                       = /etc/iet/ietd.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.722 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] image_conversion_dir           = /var/lib/cinder/conversion log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.722 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] image_upload_use_cinder_backend = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.722 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] image_upload_use_internal_tenant = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.723 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] image_volume_cache_enabled     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.723 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] image_volume_cache_max_count   = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.723 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] image_volume_cache_max_size_gb = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.724 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] instance_format                = [instance: %(uuid)s]  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.724 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] instance_uuid_format           = [instance: %(uuid)s]  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.724 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iscsi_helper                   = lioadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.725 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iscsi_iotype                   = fileio log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.725 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iscsi_ip_address               = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.725 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iscsi_port                     = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.726 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iscsi_protocol                 = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.726 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iscsi_secondary_ip_addresses   = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.726 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iscsi_target_flags             =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.727 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iscsi_target_prefix            = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.727 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iscsi_write_cache              = on log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.727 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iser_helper                    = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.728 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iser_ip_address                = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.728 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iser_port                      = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.728 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] iser_target_prefix             = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.729 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] log_config_append              = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.729 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] log_date_format                = %Y-%m-%d %H:%M:%S log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.729 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] log_dir                        = /var/log/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.730 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] log_file                       = /var/log/cinder/volume.log log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.730 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] log_options                    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.730 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] logging_context_format_string  = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.731 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] logging_debug_format_suffix    = %(funcName)s %(pathname)s:%(lineno)d log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.731 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] logging_default_format_string  = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.731 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] logging_exception_prefix       = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.732 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] logging_user_identity_format   = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.732 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] max_age                        = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.732 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] max_over_subscription_ratio    = 20.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.733 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] message_reap_interval          = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.733 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] message_ttl                    = 2592000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.733 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] migration_create_volume_timeout_secs = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.733 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] monkey_patch                   = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.734 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] monkey_patch_modules           = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.734 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] my_ip                          = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.734 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] no_snapshot_gb_quota           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.735 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] num_iser_scan_tries            = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.735 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] num_shell_tries                = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.735 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] num_volume_device_scan_tries   = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.736 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] os_privileged_user_auth_url    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.736 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] os_privileged_user_name        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.736 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] os_privileged_user_password    = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.736 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] os_privileged_user_tenant      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.737 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] osapi_max_limit                = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.737 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] osapi_volume_base_URL          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.738 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] osapi_volume_ext_list          = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.738 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.738 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] osapi_volume_listen            = controller-0.internalapi.localdomain log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.739 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] osapi_volume_listen_port       = 8776 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.739 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] osapi_volume_use_ssl           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.739 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] osapi_volume_workers           = 4 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.740 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] per_volume_size_limit          = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.740 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] periodic_fuzzy_delay           = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.740 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] periodic_interval              = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.741 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] public_endpoint                = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.741 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] publish_errors                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.741 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] query_volume_filters           = ['name', 'status', 'metadata', 'availability_zone', 'bootable', 'group_id'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.742 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] quota_backup_gigabytes         = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.742 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] quota_backups                  = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.742 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] quota_consistencygroups        = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.743 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] quota_driver                   = cinder.quota.DbQuotaDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.743 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] quota_gigabytes                = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.743 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] quota_groups                   = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.744 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] quota_snapshots                = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.744 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] quota_volumes                  = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.744 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] rate_limit_burst               = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.745 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] rate_limit_except_level        = CRITICAL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.745 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] rate_limit_interval            = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.745 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] replication_device             = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.746 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] report_discard_supported       = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.746 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] report_interval                = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.747 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] reservation_clean_interval     = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.747 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] reservation_expire             = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.747 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] reserved_percentage            = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.747 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] resource_query_filters_file    = /etc/cinder/resource_filters.json log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.748 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] rootwrap_config                = /etc/cinder/rootwrap.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.748 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] rpc_backend                    = rabbit log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.748 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] rpc_response_timeout           = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.749 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] run_external_periodic_tasks    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.749 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] scheduler_manager              = cinder.scheduler.manager.SchedulerManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.750 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] scst_target_driver             = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.750 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] scst_target_iqn_name           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.750 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] service_down_time              = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.751 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] snapshot_name_template         = snapshot-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.751 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] state_path                     = /var/lib/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.751 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] storage_availability_zone      = nova log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.752 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] storage_protocol               = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.752 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] syslog_log_facility            = LOG_USER log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.752 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] trace_flags                    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.753 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] transfer_api_class             = cinder.transfer.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.753 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] transport_url                  = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.753 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] until_refresh                  = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.754 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] use_chap_auth                  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.754 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] use_default_quota_class        = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.754 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] use_journal                    = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.755 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] use_multipath_for_image_xfer   = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.755 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] use_stderr                     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.755 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] use_syslog                     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.755 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_api_class               = cinder.volume.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.756 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_backend_name            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.756 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_clear                   = zero log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.757 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_clear_ionice            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.757 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_clear_size              = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.756 953284 ERROR cinder.volume.drivers.rbd [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:17:07.756 953284 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:17:07.756 953284 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:17:07.756 953284 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:17:07.756 953284 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:17:07.756 953284 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:17:07.756 953284 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:17:07.757 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_copy_blkio_cgroup_name  = cinder-volume-copy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.758 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_copy_bps_limit          = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.758 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_dd_blocksize            = 1M log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.758 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_manager                 = cinder.volume.manager.VolumeManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.759 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_name_template           = volume-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.759 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_service_inithost_offload = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.760 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volume_usage_audit_period      = month log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.760 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] volumes_dir                    = /etc/cinder/volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.760 953284 DEBUG cinder.utils [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:17:07.760 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] watch_log_file                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.760 953284 DEBUG cinder.utils [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Have been at this for 0.043 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:17:07.761 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] zoning_mode                    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:17:07.761 953284 DEBUG cinder.utils [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:17:07.761 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.amqp_auto_delete = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.761 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.amqp_durable_queues = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.762 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.conn_pool_min_size = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.762 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.conn_pool_ttl = 1200 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.762 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.fake_rabbit = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.763 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.heartbeat_rate = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.763 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.764 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.kombu_compression = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.764 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.kombu_failover_strategy = round-robin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.764 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.765 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.765 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_ha_queues = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.766 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_host = localhost log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.766 952691 WARNING oslo_config.cfg [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:17:07.766 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_hosts = ['localhost:5672'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.767 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_interval_max = 30 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.767 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.767 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_max_retries = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.768 952691 WARNING oslo_config.cfg [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:17:07.768 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.768 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_port = 5672 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.769 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.769 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_retry_backoff = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.769 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_retry_interval = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.770 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.770 952691 WARNING oslo_config.cfg [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:17:07.770 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_userid = guest log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.771 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rabbit_virtual_host = / log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.771 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.rpc_conn_pool_size = 30 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.771 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.ssl      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.772 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.ssl_ca_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.772 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.ssl_cert_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.772 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.ssl_key_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.773 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_rabbit.ssl_version =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.773 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] coordination.backend_url       = file:///var/lib/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.773 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] coordination.heartbeat         = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.774 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] coordination.initial_reconnect_backoff = 0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.774 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] coordination.max_reconnect_backoff = 60.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.774 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.backend               = sqlalchemy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.775 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.connection            = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.775 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.connection_debug      = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.775 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.connection_trace      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.776 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.db_inc_retry_interval = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.776 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.db_max_retries        = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.777 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.db_max_retry_interval = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.777 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.db_retry_interval     = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.777 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.idle_timeout          = 3600 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.777 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.max_overflow          = 50 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.778 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.max_pool_size         = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.778 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.max_retries           = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.778 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.min_pool_size         = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.779 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.mysql_enable_ndb      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.779 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.mysql_sql_mode        = TRADITIONAL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.779 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.pool_timeout          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.780 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.retry_interval        = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.780 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.slave_connection      = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.780 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.sqlite_synchronous    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.781 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] database.use_db_reconnect      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.781 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_notifications.driver = ['messagingv2'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.782 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_notifications.retry = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.782 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_notifications.topics = ['notifications'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.782 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_messaging_notifications.transport_url = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.783 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_reports.file_event_handler = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.783 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_reports.file_event_handler_interval = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.783 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_reports.log_dir           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.784 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.backend_availability_zone = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.784 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.backend_host      = hostgroup log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.784 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.backup_use_temp_snapshot = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.785 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.chap_password     = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.785 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.chap_username     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.785 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.chiscsi_conf      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.786 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.driver_client_cert = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.786 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.driver_client_cert_key = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.786 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.driver_data_namespace = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.787 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.driver_ssl_cert_path = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.787 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.driver_ssl_cert_verify = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.787 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.driver_use_ssl    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.788 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.enable_unsupported_driver = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.788 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.enforce_multipath_for_image_xfer = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.788 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.extra_capabilities = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.789 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.filter_function   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.789 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.goodness_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.789 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iet_conf          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.790 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.image_upload_use_cinder_backend = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.790 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.image_upload_use_internal_tenant = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.790 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.image_volume_cache_enabled = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.791 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.image_volume_cache_max_count = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.791 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.image_volume_cache_max_size_gb = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.791 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iscsi_helper      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.792 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iscsi_iotype      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.792 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iscsi_ip_address  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.792 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iscsi_port        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.793 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iscsi_protocol    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.793 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iscsi_secondary_ip_addresses = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.793 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iscsi_target_flags = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.794 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iscsi_target_prefix = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.794 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iscsi_write_cache = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.794 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iser_helper       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.795 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iser_ip_address   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.795 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iser_port         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.795 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.iser_target_prefix = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.795 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.max_over_subscription_ratio = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.796 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.num_iser_scan_tries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.796 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.num_shell_tries   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.796 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.num_volume_device_scan_tries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.797 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rados_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.797 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rados_connection_interval = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.797 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rados_connection_retries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.798 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rbd_ceph_conf     = /etc/ceph/ceph.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.798 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rbd_cluster_name  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.798 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rbd_flatten_volume_from_snapshot = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.799 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rbd_keyring_conf  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.799 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rbd_max_clone_depth = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.799 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rbd_pool          = volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.799 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rbd_secret_uuid   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.800 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rbd_store_chunk_size = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.800 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.rbd_user          = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.800 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.replication_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.801 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.replication_device = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.801 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.report_discard_supported = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.801 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.report_dynamic_total_capacity = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.802 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.reserved_percentage = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.802 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.scst_target_driver = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.802 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.scst_target_iqn_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.802 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.storage_protocol  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.803 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.suppress_requests_ssl_warnings = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.803 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.trace_flags       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.803 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.use_chap_auth     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.804 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.use_multipath_for_image_xfer = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.804 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.volume_backend_name = tripleo_ceph log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.804 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.volume_clear      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.805 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.volume_clear_ionice = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.805 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.volume_clear_size = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.805 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.volume_copy_blkio_cgroup_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.805 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.volume_copy_bps_limit = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.806 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.volume_dd_blocksize = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.806 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.volume_driver     = cinder.volume.drivers.rbd.RBDDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.806 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] tripleo_ceph.volumes_dir       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.807 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] profiler.connection_string     = messaging:// log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.807 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] profiler.enabled               = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.807 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] profiler.es_doc_type           = notification log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.808 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] profiler.es_scroll_size        = 10000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.808 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] profiler.es_scroll_time        = 2m log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.808 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] profiler.hmac_keys             = SECRET_KEY log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.808 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] profiler.sentinel_service_name = mymaster log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.809 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] profiler.socket_timeout        = 0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.809 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] profiler.trace_sqlalchemy      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.809 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] privsep_osbrick.capabilities   = [21] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.810 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] privsep_osbrick.group          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.810 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] privsep_osbrick.helper_command = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.810 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] privsep_osbrick.user           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.811 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.backend_availability_zone = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.811 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.backup_use_temp_snapshot = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.811 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.chap_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.812 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.chap_username =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.812 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.812 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.driver_client_cert = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.813 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.driver_client_cert_key = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.813 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.driver_data_namespace = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.813 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.driver_ssl_cert_path = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.813 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.driver_ssl_cert_verify = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.814 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.driver_use_ssl = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.814 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.enable_unsupported_driver = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.814 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.enforce_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.815 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.extra_capabilities = {} log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.815 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.filter_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.815 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.goodness_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.816 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iet_conf      = /etc/iet/ietd.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.816 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.image_upload_use_cinder_backend = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.816 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.image_upload_use_internal_tenant = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.816 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.image_volume_cache_enabled = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.817 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.image_volume_cache_max_count = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.817 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.image_volume_cache_max_size_gb = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.817 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iscsi_helper  = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.818 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iscsi_iotype  = fileio log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.818 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iscsi_ip_address = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.819 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iscsi_port    = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.819 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iscsi_protocol = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.819 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iscsi_secondary_ip_addresses = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.820 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iscsi_target_flags =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.820 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iscsi_target_prefix = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.820 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iscsi_write_cache = on log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.821 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iser_helper   = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.821 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iser_ip_address = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.821 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iser_port     = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.822 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.822 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.max_over_subscription_ratio = 20.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.822 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.num_iser_scan_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.823 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.num_shell_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.823 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.num_volume_device_scan_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.823 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rados_connect_timeout = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.824 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rados_connection_interval = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.824 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rados_connection_retries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.824 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rbd_ceph_conf =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.825 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rbd_cluster_name = ceph log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.825 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rbd_flatten_volume_from_snapshot = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.825 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rbd_keyring_conf =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.826 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rbd_max_clone_depth = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.826 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rbd_pool      = rbd log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.826 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rbd_secret_uuid = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.827 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rbd_store_chunk_size = 4 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.827 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.rbd_user      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.827 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.replication_connect_timeout = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.828 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.replication_device = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.828 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.report_discard_supported = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.828 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.report_dynamic_total_capacity = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.829 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.reserved_percentage = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.829 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.scst_target_driver = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.829 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.scst_target_iqn_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.830 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.storage_protocol = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.830 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.suppress_requests_ssl_warnings = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.831 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.trace_flags   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.831 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.use_chap_auth = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.831 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.use_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.832 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.volume_backend_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.832 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.volume_clear  = zero log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.832 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.volume_clear_ionice = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.833 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.volume_clear_size = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.833 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.833 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.volume_copy_bps_limit = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.834 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.volume_dd_blocksize = 1M log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.834 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.835 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] backend_defaults.volumes_dir   = /var/lib/cinder/volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.835 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_concurrency.disable_process_locking = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.836 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_concurrency.lock_path     = /var/lib/cinder/tmp log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.836 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] barbican.auth_endpoint         = http://localhost/identity/v3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.836 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] barbican.barbican_api_version  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.837 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] barbican.barbican_endpoint     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.837 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] barbican.number_of_retries     = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.837 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] barbican.retry_delay           = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.838 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] barbican.verify_ssl            = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.838 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.admin_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.838 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.admin_tenant_name = admin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.839 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.admin_token = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.839 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.admin_user  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.840 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.auth_admin_prefix =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.840 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.auth_host   = 127.0.0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.840 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.auth_port   = 35357 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.841 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.auth_protocol = http log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.841 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.auth_section = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.841 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.auth_type   = password log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.842 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.auth_uri    = http://172.17.1.12:5000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.842 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.auth_version = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.842 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.cache       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.843 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.cafile      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.843 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.certfile    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.843 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.check_revocations_for_cached = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.844 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.delay_auth_decision = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.844 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.enforce_token_bind = permissive log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.844 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.hash_algorithms = ['md5'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.845 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.http_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.845 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.http_request_max_retries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.845 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.identity_uri = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.846 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.include_service_catalog = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.846 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.insecure    = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.846 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.keyfile     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.847 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.memcache_pool_conn_get_timeout = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.847 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.memcache_pool_dead_retry = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.847 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.memcache_pool_maxsize = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.848 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.memcache_pool_socket_timeout = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.848 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.memcache_pool_unused_timeout = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.848 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.memcache_secret_key = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.849 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.memcache_security_strategy = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.849 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.memcache_use_advanced_pool = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.849 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.memcached_servers = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.850 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.region_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.850 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.revocation_cache_time = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.850 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.service_token_roles = ['service'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.851 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.service_token_roles_required = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.851 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.signing_dir = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.851 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] keystone_authtoken.token_cache_time = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.852 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_versionedobjects.fatal_exception_format_errors = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.852 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_policy.policy_default_rule = default log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.852 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_policy.policy_dirs        = ['policy.d'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.853 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] oslo_policy.policy_file        = /etc/cinder/policy.json log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.853 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] key_manager.api_class          = cinder.keymgr.conf_key_mgr.ConfKeyManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.853 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] key_manager.fixed_key          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:17:07.854 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2889
>2018-06-25 08:17:17.761 953284 DEBUG cinder.volume.drivers.rbd [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:17:17.793 953284 ERROR cinder.volume.drivers.rbd [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:17:17.793 953284 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:17:17.793 953284 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:17:17.793 953284 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:17:17.793 953284 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:17:17.793 953284 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:17:17.793 953284 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:17:17.795 953284 DEBUG cinder.utils [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:17:17.796 953284 DEBUG cinder.utils [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Have been at this for 10.079 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:17:17.796 953284 DEBUG cinder.utils [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:17:37.796 953284 DEBUG cinder.volume.drivers.rbd [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:17:37.832 953284 ERROR cinder.volume.drivers.rbd [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:17:37.832 953284 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:17:37.832 953284 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:17:37.832 953284 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:17:37.832 953284 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:17:37.832 953284 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:17:37.832 953284 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:17:37.834 953284 DEBUG cinder.utils [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:17:37.834 953284 DEBUG cinder.utils [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Have been at this for 30.117 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Failed to initialize driver.: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 431, in init_host
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager     self.driver.check_for_setup_error()
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 272, in check_for_setup_error
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager     with RADOSClient(self):
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager     return _do_conn(pool, remote, timeout)
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager     return r.call(f, *args, **kwargs)
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager     raise attempt.get()
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:17:37.834 953284 ERROR cinder.volume.manager 
>2018-06-25 08:17:37.837 953284 DEBUG cinder.service [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Creating RPC server for service cinder-volume start /usr/lib/python2.7/site-packages/cinder/service.py:219
>2018-06-25 08:17:37.846 953284 DEBUG oslo_db.sqlalchemy.engines [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:17:37.854 953284 DEBUG cinder.service [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Pinning object versions for RPC server serializer to 1.27 start /usr/lib/python2.7/site-packages/cinder/service.py:226
>2018-06-25 08:17:37.857 953284 WARNING oslo_config.cfg [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:17:37.857 953284 WARNING oslo_config.cfg [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:17:37.858 953284 WARNING oslo_config.cfg [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:17:37.981 953284 INFO cinder.volume.manager [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Initializing RPC dependent components of volume driver RBDDriver (1.2.0)
>2018-06-25 08:17:37.982 953284 ERROR cinder.utils [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Volume driver RBDDriver not initialized
>2018-06-25 08:17:37.982 953284 ERROR cinder.volume.manager [req-0a6226b6-57a8-4a1a-b7b7-4a9893357f67 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready.
>2018-06-25 08:17:47.984 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:17:57.985 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:18:07.986 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:18:17.987 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:18:27.987 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:18:37.989 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:18:47.990 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:18:57.991 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:18:59.986 953284 DEBUG oslo_service.periodic_task [req-9f256c52-2459-49bd-8545-6eaedb05fcb0 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:18:59.987 953284 DEBUG oslo_service.periodic_task [req-9f256c52-2459-49bd-8545-6eaedb05fcb0 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:18:59.987 953284 WARNING cinder.volume.manager [req-9f256c52-2459-49bd-8545-6eaedb05fcb0 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:19:07.993 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:19:17.994 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:19:27.994 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:19:37.996 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:19:47.996 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:19:57.997 953284 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:19:59.988 953284 DEBUG oslo_service.periodic_task [req-9f256c52-2459-49bd-8545-6eaedb05fcb0 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:19:59.989 953284 DEBUG oslo_service.periodic_task [req-9f256c52-2459-49bd-8545-6eaedb05fcb0 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:19:59.989 953284 WARNING cinder.volume.manager [req-9f256c52-2459-49bd-8545-6eaedb05fcb0 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:20:05.791 952691 INFO oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Caught SIGTERM, stopping children
>2018-06-25 08:20:05.793 952691 DEBUG oslo_concurrency.lockutils [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 08:20:05.793 952691 DEBUG oslo_concurrency.lockutils [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 08:20:05.794 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Stop services. stop /usr/lib/python2.7/site-packages/oslo_service/service.py:624
>2018-06-25 08:20:05.794 952691 DEBUG oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Killing children. stop /usr/lib/python2.7/site-packages/oslo_service/service.py:629
>2018-06-25 08:20:05.794 952691 INFO oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Waiting on 1 children to exit
>2018-06-25 08:20:07.984 953284 DEBUG oslo_concurrency.lockutils [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 08:20:07.985 953284 DEBUG oslo_concurrency.lockutils [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 08:21:00.029 952691 INFO oslo_service.service [req-7cd3520b-fc85-4e64-8372-2c4b11bdfdea - - - - -] Child 953284 exited with status 0
>2018-06-25 08:21:03.854 980023 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
>2018-06-25 08:21:03.855 980023 DEBUG oslo_concurrency.lockutils [-] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 08:21:03.855 980023 DEBUG oslo_concurrency.lockutils [-] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 08:21:04.246 980023 DEBUG oslo_db.sqlalchemy.engines [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:21:04.402 980023 INFO cinder.rpc [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version.
>2018-06-25 08:21:04.408 980023 INFO cinder.rpc [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version.
>2018-06-25 08:21:04.415 980023 DEBUG oslo_db.sqlalchemy.engines [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:21:04.424 980023 WARNING cinder.keymgr.conf_key_mgr [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] This key manager is insecure and is not recommended for production deployments
>2018-06-25 08:21:04.434 980023 INFO cinder.volume.manager [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Image-volume cache disabled for host hostgroup@tripleo_ceph.
>2018-06-25 08:21:04.439 980023 DEBUG oslo_db.sqlalchemy.engines [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:21:04.451 980023 INFO oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Starting 1 workers
>2018-06-25 08:21:04.455 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Started child 980119 _start_child /usr/lib/python2.7/site-packages/oslo_service/service.py:513
>2018-06-25 08:21:04.458 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Full set of CONF: wait /usr/lib/python2.7/site-packages/oslo_service/service.py:580
>2018-06-25 08:21:04.459 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2865
>2018-06-25 08:21:04.459 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Configuration options gathered from: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2866
>2018-06-25 08:21:04.460 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] command line args: ['--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--logfile', '/var/log/cinder/volume.log'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2867
>2018-06-25 08:21:04.460 980119 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0)
>2018-06-25 08:21:04.460 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] config files: ['/usr/share/cinder/cinder-dist.conf', '/etc/cinder/cinder.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2869
>2018-06-25 08:21:04.461 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] ================================================================================ log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2870
>2018-06-25 08:21:04.461 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] allow_availability_zone_fallback = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.462 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] allowed_direct_url_schemes     = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.462 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] api_rate_limit                 = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.463 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] auth_strategy                  = keystone log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.463 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_availability_zone      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.464 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_host                   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.464 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backup_api_class               = cinder.backup.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.464 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backup_manager                 = cinder.backup.manager.BackupManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.465 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backup_name_template           = backup-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.465 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backup_use_same_host           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.466 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backup_use_temp_snapshot       = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.466 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] chap_password                  = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.466 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] chap_username                  =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.467 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.467 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] cinder_internal_tenant_project_id = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.468 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] cinder_internal_tenant_user_id = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.468 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] cluster                        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.469 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] compute_api_class              = cinder.compute.nova.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.469 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] config_dir                     = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.470 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] config_file                    = ['/usr/share/cinder/cinder-dist.conf', '/etc/cinder/cinder.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.470 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] consistencygroup_api_class     = cinder.consistencygroup.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.471 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] control_exchange               = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.471 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] db_driver                      = cinder.db log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.472 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] debug                          = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.472 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] default_availability_zone      = nova log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.473 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] default_group_type             = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.473 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'dogpile.core.dogpile=INFO'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.474 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] default_volume_type            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.474 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] driver_client_cert             = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.474 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] driver_client_cert_key         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.475 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] driver_data_namespace          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.475 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] driver_ssl_cert_path           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.476 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] driver_ssl_cert_verify         = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.476 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] driver_use_ssl                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.476 980119 INFO cinder.volume.manager [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Starting volume driver RBDDriver (1.2.0)
>2018-06-25 08:21:04.476 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] enable_new_services            = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.477 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] enable_unsupported_driver      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.477 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] enable_v1_api                  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.477 980119 DEBUG cinder.volume.drivers.rbd [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:21:04.477 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] enable_v2_api                  = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.478 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] enable_v3_api                  = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.478 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] enabled_backends               = ['tripleo_ceph'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.479 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] enforce_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.479 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] fatal_exception_format_errors  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.480 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] filter_function                = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.480 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] glance_api_insecure            = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.480 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] glance_api_servers             = ['http://172.17.1.12:9292'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.481 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] glance_api_ssl_compression     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.481 980023 WARNING oslo_config.cfg [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens).  Its value may be silently ignored in the future.
>2018-06-25 08:21:04.482 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] glance_api_version             = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.482 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] glance_ca_certificates_file    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.482 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] glance_catalog_info            = image:glance:publicURL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.483 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.484 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] glance_num_retries             = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.484 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] glance_request_timeout         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.484 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] goodness_function              = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.485 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] graceful_shutdown_timeout      = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.485 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] group_api_class                = cinder.group.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.486 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] host                           = hostgroup log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.487 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iet_conf                       = /etc/iet/ietd.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.487 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] image_conversion_dir           = /var/lib/cinder/conversion log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.488 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] image_upload_use_cinder_backend = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.488 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] image_upload_use_internal_tenant = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.488 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] image_volume_cache_enabled     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.489 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] image_volume_cache_max_count   = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.489 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] image_volume_cache_max_size_gb = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.490 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] instance_format                = [instance: %(uuid)s]  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.490 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] instance_uuid_format           = [instance: %(uuid)s]  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.490 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iscsi_helper                   = lioadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.491 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iscsi_iotype                   = fileio log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.491 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iscsi_ip_address               = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.492 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iscsi_port                     = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.492 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iscsi_protocol                 = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.492 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iscsi_secondary_ip_addresses   = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.493 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iscsi_target_flags             =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.493 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iscsi_target_prefix            = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.494 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iscsi_write_cache              = on log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.494 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iser_helper                    = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.494 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iser_ip_address                = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.495 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iser_port                      = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.495 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] iser_target_prefix             = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.495 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] log_config_append              = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.496 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] log_date_format                = %Y-%m-%d %H:%M:%S log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.496 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] log_dir                        = /var/log/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.497 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] log_file                       = /var/log/cinder/volume.log log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.497 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] log_options                    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.497 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] logging_context_format_string  = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.498 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] logging_debug_format_suffix    = %(funcName)s %(pathname)s:%(lineno)d log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.498 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] logging_default_format_string  = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.498 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] logging_exception_prefix       = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.499 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] logging_user_identity_format   = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.499 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] max_age                        = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.499 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] max_over_subscription_ratio    = 20.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.500 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] message_reap_interval          = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.500 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] message_ttl                    = 2592000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.501 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] migration_create_volume_timeout_secs = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.501 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] monkey_patch                   = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.501 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] monkey_patch_modules           = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.502 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] my_ip                          = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.502 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] no_snapshot_gb_quota           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.502 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] num_iser_scan_tries            = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.503 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] num_shell_tries                = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.503 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] num_volume_device_scan_tries   = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.503 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] os_privileged_user_auth_url    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.504 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] os_privileged_user_name        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.504 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] os_privileged_user_password    = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.504 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] os_privileged_user_tenant      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.505 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] osapi_max_limit                = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.505 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] osapi_volume_base_URL          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.505 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] osapi_volume_ext_list          = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.506 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.506 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] osapi_volume_listen            = controller-0.internalapi.localdomain log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.507 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] osapi_volume_listen_port       = 8776 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.507 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] osapi_volume_use_ssl           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.507 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] osapi_volume_workers           = 4 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.508 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] per_volume_size_limit          = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.508 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] periodic_fuzzy_delay           = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.508 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] periodic_interval              = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.509 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] public_endpoint                = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.509 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] publish_errors                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.510 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] query_volume_filters           = ['name', 'status', 'metadata', 'availability_zone', 'bootable', 'group_id'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.510 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] quota_backup_gigabytes         = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.510 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] quota_backups                  = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.511 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] quota_consistencygroups        = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.511 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] quota_driver                   = cinder.quota.DbQuotaDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.512 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] quota_gigabytes                = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.512 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] quota_groups                   = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.512 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] quota_snapshots                = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.513 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] quota_volumes                  = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.513 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] rate_limit_burst               = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.513 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] rate_limit_except_level        = CRITICAL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.514 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] rate_limit_interval            = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.514 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] replication_device             = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.514 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] report_discard_supported       = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.515 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] report_interval                = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.516 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] reservation_clean_interval     = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.516 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] reservation_expire             = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.516 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] reserved_percentage            = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.517 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] resource_query_filters_file    = /etc/cinder/resource_filters.json log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.517 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] rootwrap_config                = /etc/cinder/rootwrap.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.518 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] rpc_backend                    = rabbit log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.518 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] rpc_response_timeout           = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.518 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] run_external_periodic_tasks    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.517 980119 ERROR cinder.volume.drivers.rbd [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:21:04.517 980119 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:21:04.517 980119 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:21:04.517 980119 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:21:04.517 980119 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:21:04.517 980119 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:21:04.517 980119 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:21:04.519 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] scheduler_manager              = cinder.scheduler.manager.SchedulerManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.519 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] scst_target_driver             = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.519 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] scst_target_iqn_name           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.520 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] service_down_time              = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.520 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] snapshot_name_template         = snapshot-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.520 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] state_path                     = /var/lib/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.521 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] storage_availability_zone      = nova log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.521 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] storage_protocol               = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.521 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] syslog_log_facility            = LOG_USER log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.521 980119 DEBUG cinder.utils [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:21:04.522 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] trace_flags                    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.522 980119 DEBUG cinder.utils [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Have been at this for 0.045 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:21:04.522 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] transfer_api_class             = cinder.transfer.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.522 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] transport_url                  = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.522 980119 DEBUG cinder.utils [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:21:04.523 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] until_refresh                  = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.523 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] use_chap_auth                  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.523 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] use_default_quota_class        = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.524 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] use_journal                    = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.524 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] use_multipath_for_image_xfer   = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.524 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] use_stderr                     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.525 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] use_syslog                     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.525 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_api_class               = cinder.volume.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.525 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_backend_name            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.525 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_clear                   = zero log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.526 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_clear_ionice            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.526 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_clear_size              = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.527 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_copy_blkio_cgroup_name  = cinder-volume-copy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.527 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_copy_bps_limit          = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.528 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_dd_blocksize            = 1M log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.528 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_manager                 = cinder.volume.manager.VolumeManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.528 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_name_template           = volume-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.529 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_service_inithost_offload = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.529 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volume_usage_audit_period      = month log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.529 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] volumes_dir                    = /etc/cinder/volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.530 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] watch_log_file                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.530 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] zoning_mode                    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 08:21:04.530 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.amqp_auto_delete = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.531 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.amqp_durable_queues = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.531 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.conn_pool_min_size = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.532 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.conn_pool_ttl = 1200 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.532 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.fake_rabbit = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.533 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.heartbeat_rate = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.533 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.534 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.kombu_compression = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.534 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.kombu_failover_strategy = round-robin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.534 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.535 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.535 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_ha_queues = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.536 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_host = localhost log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.536 980023 WARNING oslo_config.cfg [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:21:04.537 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_hosts = ['localhost:5672'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.537 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_interval_max = 30 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.538 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.538 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_max_retries = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.538 980023 WARNING oslo_config.cfg [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:21:04.539 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.539 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_port = 5672 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.539 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.540 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_retry_backoff = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.540 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_retry_interval = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.541 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.541 980023 WARNING oslo_config.cfg [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:21:04.541 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_userid = guest log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.542 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rabbit_virtual_host = / log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.542 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.rpc_conn_pool_size = 30 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.542 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.ssl      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.543 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.ssl_ca_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.543 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.ssl_cert_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.544 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.ssl_key_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.544 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_rabbit.ssl_version =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.544 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] coordination.backend_url       = file:///var/lib/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.545 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] coordination.heartbeat         = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.545 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] coordination.initial_reconnect_backoff = 0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.546 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] coordination.max_reconnect_backoff = 60.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.546 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.backend               = sqlalchemy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.547 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.connection            = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.547 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.connection_debug      = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.547 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.connection_trace      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.548 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.db_inc_retry_interval = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.548 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.db_max_retries        = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.548 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.db_max_retry_interval = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.549 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.db_retry_interval     = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.549 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.idle_timeout          = 3600 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.550 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.max_overflow          = 50 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.550 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.max_pool_size         = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.550 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.max_retries           = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.551 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.min_pool_size         = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.551 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.mysql_enable_ndb      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.551 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.mysql_sql_mode        = TRADITIONAL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.552 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.pool_timeout          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.552 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.retry_interval        = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.552 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.slave_connection      = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.553 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.sqlite_synchronous    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.553 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] database.use_db_reconnect      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.554 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_notifications.driver = ['messagingv2'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.554 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_notifications.retry = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.554 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_notifications.topics = ['notifications'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.555 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_messaging_notifications.transport_url = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.555 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_reports.file_event_handler = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.556 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_reports.file_event_handler_interval = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.556 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_reports.log_dir           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.556 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.backend_availability_zone = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.557 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.backend_host      = hostgroup log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.557 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.backup_use_temp_snapshot = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.557 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.chap_password     = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.558 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.chap_username     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.558 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.chiscsi_conf      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.558 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.driver_client_cert = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.559 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.driver_client_cert_key = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.559 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.driver_data_namespace = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.559 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.driver_ssl_cert_path = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.560 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.driver_ssl_cert_verify = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.560 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.driver_use_ssl    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.560 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.enable_unsupported_driver = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.561 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.enforce_multipath_for_image_xfer = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.561 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.extra_capabilities = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.561 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.filter_function   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.562 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.goodness_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.562 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iet_conf          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.562 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.image_upload_use_cinder_backend = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.563 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.image_upload_use_internal_tenant = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.563 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.image_volume_cache_enabled = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.563 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.image_volume_cache_max_count = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.564 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.image_volume_cache_max_size_gb = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.564 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iscsi_helper      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.564 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iscsi_iotype      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.565 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iscsi_ip_address  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.565 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iscsi_port        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.565 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iscsi_protocol    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.566 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iscsi_secondary_ip_addresses = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.566 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iscsi_target_flags = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.566 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iscsi_target_prefix = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.567 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iscsi_write_cache = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.567 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iser_helper       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.567 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iser_ip_address   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.568 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iser_port         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.568 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.iser_target_prefix = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.568 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.max_over_subscription_ratio = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.569 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.num_iser_scan_tries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.569 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.num_shell_tries   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.569 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.num_volume_device_scan_tries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.570 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rados_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.570 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rados_connection_interval = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.570 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rados_connection_retries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.571 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rbd_ceph_conf     = /etc/ceph/ceph.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.571 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rbd_cluster_name  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.571 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rbd_flatten_volume_from_snapshot = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.572 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rbd_keyring_conf  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.572 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rbd_max_clone_depth = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.572 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rbd_pool          = volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.573 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rbd_secret_uuid   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.573 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rbd_store_chunk_size = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.573 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.rbd_user          = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.574 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.replication_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.574 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.replication_device = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.574 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.report_discard_supported = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.575 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.report_dynamic_total_capacity = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.575 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.reserved_percentage = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.575 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.scst_target_driver = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.576 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.scst_target_iqn_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.576 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.storage_protocol  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.576 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.suppress_requests_ssl_warnings = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.577 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.trace_flags       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.577 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.use_chap_auth     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.577 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.use_multipath_for_image_xfer = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.578 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.volume_backend_name = tripleo_ceph log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.578 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.volume_clear      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.578 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.volume_clear_ionice = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.579 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.volume_clear_size = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.579 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.volume_copy_blkio_cgroup_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.579 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.volume_copy_bps_limit = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.580 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.volume_dd_blocksize = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.580 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.volume_driver     = cinder.volume.drivers.rbd.RBDDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.580 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] tripleo_ceph.volumes_dir       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.581 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] profiler.connection_string     = messaging:// log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.581 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] profiler.enabled               = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.581 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] profiler.es_doc_type           = notification log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.582 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] profiler.es_scroll_size        = 10000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.582 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] profiler.es_scroll_time        = 2m log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.583 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] profiler.hmac_keys             = SECRET_KEY log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.583 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] profiler.sentinel_service_name = mymaster log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.583 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] profiler.socket_timeout        = 0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.584 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] profiler.trace_sqlalchemy      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.584 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] privsep_osbrick.capabilities   = [21] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.584 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] privsep_osbrick.group          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.585 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] privsep_osbrick.helper_command = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.585 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] privsep_osbrick.user           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.585 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.backend_availability_zone = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.586 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.backup_use_temp_snapshot = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.586 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.chap_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.587 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.chap_username =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.587 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.587 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.driver_client_cert = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.588 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.driver_client_cert_key = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.588 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.driver_data_namespace = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.588 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.driver_ssl_cert_path = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.589 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.driver_ssl_cert_verify = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.589 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.driver_use_ssl = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.589 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.enable_unsupported_driver = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.590 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.enforce_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.590 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.extra_capabilities = {} log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.590 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.filter_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.591 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.goodness_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.591 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iet_conf      = /etc/iet/ietd.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.591 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.image_upload_use_cinder_backend = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.592 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.image_upload_use_internal_tenant = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.592 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.image_volume_cache_enabled = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.592 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.image_volume_cache_max_count = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.593 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.image_volume_cache_max_size_gb = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.593 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iscsi_helper  = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.593 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iscsi_iotype  = fileio log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.594 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iscsi_ip_address = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.594 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iscsi_port    = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.595 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iscsi_protocol = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.595 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iscsi_secondary_ip_addresses = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.595 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iscsi_target_flags =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.596 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iscsi_target_prefix = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.596 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iscsi_write_cache = on log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.596 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iser_helper   = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.597 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iser_ip_address = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.597 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iser_port     = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.598 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.598 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.max_over_subscription_ratio = 20.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.598 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.num_iser_scan_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.599 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.num_shell_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.599 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.num_volume_device_scan_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.599 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rados_connect_timeout = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.600 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rados_connection_interval = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.600 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rados_connection_retries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.600 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rbd_ceph_conf =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.601 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rbd_cluster_name = ceph log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.601 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rbd_flatten_volume_from_snapshot = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.601 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rbd_keyring_conf =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.602 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rbd_max_clone_depth = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.602 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rbd_pool      = rbd log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.602 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rbd_secret_uuid = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.603 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rbd_store_chunk_size = 4 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.603 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.rbd_user      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.603 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.replication_connect_timeout = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.604 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.replication_device = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.604 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.report_discard_supported = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.604 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.report_dynamic_total_capacity = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.605 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.reserved_percentage = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.605 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.scst_target_driver = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.605 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.scst_target_iqn_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.606 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.storage_protocol = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.606 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.suppress_requests_ssl_warnings = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.606 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.trace_flags   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.607 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.use_chap_auth = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.607 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.use_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.607 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.volume_backend_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.608 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.volume_clear  = zero log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.608 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.volume_clear_ionice = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.608 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.volume_clear_size = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.609 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.609 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.volume_copy_bps_limit = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.609 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.volume_dd_blocksize = 1M log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.610 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.610 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] backend_defaults.volumes_dir   = /var/lib/cinder/volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.611 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_concurrency.disable_process_locking = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.611 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_concurrency.lock_path     = /var/lib/cinder/tmp log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.612 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] barbican.auth_endpoint         = http://localhost/identity/v3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.612 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] barbican.barbican_api_version  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.612 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] barbican.barbican_endpoint     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.613 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] barbican.number_of_retries     = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.613 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] barbican.retry_delay           = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.613 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] barbican.verify_ssl            = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.614 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.admin_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.614 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.admin_tenant_name = admin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.615 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.admin_token = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.615 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.admin_user  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.615 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.auth_admin_prefix =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.616 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.auth_host   = 127.0.0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.616 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.auth_port   = 35357 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.616 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.auth_protocol = http log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.617 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.auth_section = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.617 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.auth_type   = password log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.618 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.auth_uri    = http://172.17.1.12:5000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.618 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.auth_version = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.618 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.cache       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.619 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.cafile      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.619 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.certfile    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.619 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.check_revocations_for_cached = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.620 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.delay_auth_decision = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.620 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.enforce_token_bind = permissive log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.620 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.hash_algorithms = ['md5'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.621 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.http_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.621 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.http_request_max_retries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.621 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.identity_uri = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.622 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.include_service_catalog = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.622 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.insecure    = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.622 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.keyfile     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.623 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.memcache_pool_conn_get_timeout = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.623 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.memcache_pool_dead_retry = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.623 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.memcache_pool_maxsize = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.624 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.memcache_pool_socket_timeout = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.624 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.memcache_pool_unused_timeout = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.624 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.memcache_secret_key = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.625 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.memcache_security_strategy = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.625 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.memcache_use_advanced_pool = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.625 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.memcached_servers = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.626 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.region_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.626 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.revocation_cache_time = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.626 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.service_token_roles = ['service'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.627 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.service_token_roles_required = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.627 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.signing_dir = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.628 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] keystone_authtoken.token_cache_time = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.628 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_versionedobjects.fatal_exception_format_errors = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.628 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_policy.policy_default_rule = default log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.629 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_policy.policy_dirs        = ['policy.d'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.629 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] oslo_policy.policy_file        = /etc/cinder/policy.json log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.629 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] key_manager.api_class          = cinder.keymgr.conf_key_mgr.ConfKeyManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.630 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] key_manager.fixed_key          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 08:21:04.630 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2889
>2018-06-25 08:21:14.523 980119 DEBUG cinder.volume.drivers.rbd [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:21:14.557 980119 ERROR cinder.volume.drivers.rbd [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:21:14.557 980119 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:21:14.557 980119 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:21:14.557 980119 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:21:14.557 980119 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:21:14.557 980119 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:21:14.557 980119 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:21:14.559 980119 DEBUG cinder.utils [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:21:14.560 980119 DEBUG cinder.utils [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Have been at this for 10.083 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:21:14.560 980119 DEBUG cinder.utils [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 08:21:34.561 980119 DEBUG cinder.volume.drivers.rbd [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 08:21:34.596 980119 ERROR cinder.volume.drivers.rbd [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 08:21:34.596 980119 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 08:21:34.596 980119 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 08:21:34.596 980119 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 08:21:34.596 980119 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 08:21:34.596 980119 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 08:21:34.596 980119 ERROR cinder.volume.drivers.rbd 
>2018-06-25 08:21:34.598 980119 DEBUG cinder.utils [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 08:21:34.598 980119 DEBUG cinder.utils [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Failed to initialize driver.: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 431, in init_host
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager     self.driver.check_for_setup_error()
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 272, in check_for_setup_error
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager     with RADOSClient(self):
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager     return _do_conn(pool, remote, timeout)
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager     return r.call(f, *args, **kwargs)
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager     raise attempt.get()
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 08:21:34.599 980119 ERROR cinder.volume.manager 
>2018-06-25 08:21:34.601 980119 DEBUG cinder.service [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Creating RPC server for service cinder-volume start /usr/lib/python2.7/site-packages/cinder/service.py:219
>2018-06-25 08:21:34.609 980119 DEBUG oslo_db.sqlalchemy.engines [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 08:21:34.616 980119 DEBUG cinder.service [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Pinning object versions for RPC server serializer to 1.27 start /usr/lib/python2.7/site-packages/cinder/service.py:226
>2018-06-25 08:21:34.618 980119 WARNING oslo_config.cfg [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:21:34.619 980119 WARNING oslo_config.cfg [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:21:34.619 980119 WARNING oslo_config.cfg [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 08:21:34.731 980119 INFO cinder.volume.manager [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Initializing RPC dependent components of volume driver RBDDriver (1.2.0)
>2018-06-25 08:21:34.732 980119 ERROR cinder.utils [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Volume driver RBDDriver not initialized
>2018-06-25 08:21:34.732 980119 ERROR cinder.volume.manager [req-578b4983-2de6-4e1c-8f6a-64189aacb55a - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready.
>2018-06-25 08:21:44.733 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:21:54.734 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:22:04.736 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:22:09.734 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:22:09.735 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:22:09.735 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:22:14.737 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:22:24.738 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:22:34.740 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:22:44.741 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:22:54.741 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:23:04.743 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:23:09.737 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:23:09.738 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:23:09.738 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:23:14.744 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:23:24.744 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:23:34.745 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:23:44.746 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:23:54.747 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:24:04.748 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:24:09.739 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:24:09.740 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:24:09.740 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:24:14.749 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:24:24.749 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:24:34.750 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:24:44.750 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:24:54.751 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:25:04.753 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:25:09.741 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:25:09.742 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:25:09.743 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:25:14.754 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:25:24.754 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:25:34.755 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:25:44.756 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:25:54.756 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:26:04.757 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:26:09.743 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:26:09.744 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:26:09.745 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:26:14.762 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:26:24.763 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:26:34.765 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:26:44.766 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:26:54.766 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:27:04.768 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:27:09.746 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:27:14.769 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:27:24.770 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:27:34.771 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:27:44.772 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:27:54.773 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:28:04.774 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:28:09.747 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:28:09.748 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:28:14.775 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:28:24.776 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:28:34.777 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:28:44.778 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:28:54.779 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:29:04.780 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:29:09.749 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:29:09.750 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:29:09.751 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:29:14.781 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:29:24.782 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:29:34.782 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:29:44.783 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:29:54.784 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:30:04.785 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:30:09.752 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:30:09.753 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:30:09.753 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:30:14.786 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:30:24.786 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:30:34.787 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:30:44.788 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:30:54.788 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:31:04.790 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:31:09.755 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:31:09.755 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:31:09.756 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:31:14.790 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:31:24.791 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:31:34.792 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:31:44.793 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:31:54.793 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:32:04.794 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:32:09.757 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:32:09.757 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:32:09.758 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:32:14.795 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:32:24.796 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:32:34.796 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:32:44.797 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:32:54.798 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:33:04.799 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:33:09.759 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:33:09.759 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:33:09.760 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:33:14.799 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:33:24.801 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:33:34.803 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:33:44.803 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:33:54.804 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:34:04.805 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:34:09.761 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:34:09.762 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:34:09.762 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:34:14.806 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:34:24.807 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:34:34.807 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:34:44.808 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:34:54.809 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:35:04.810 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:35:09.763 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:35:09.764 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:35:09.764 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:35:14.811 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:35:24.811 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:35:34.812 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:35:44.813 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:35:54.814 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:36:04.815 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:36:09.765 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:36:09.766 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:36:09.766 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:36:14.815 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:36:24.816 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:36:34.817 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:36:44.818 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:36:54.819 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:37:04.820 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:37:09.767 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:37:09.768 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:37:09.768 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:37:14.820 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:37:24.821 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:37:34.822 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:37:44.822 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:37:54.823 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:38:04.824 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:38:09.769 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:38:09.770 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:38:09.770 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:38:14.825 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:38:24.826 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:38:34.826 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:38:44.827 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:38:54.828 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:39:04.829 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:39:09.771 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:39:09.772 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:39:09.772 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:39:14.829 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:39:24.830 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:39:34.830 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:39:44.831 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:39:54.833 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:40:04.833 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:40:09.773 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:40:09.774 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:40:09.774 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:40:14.834 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:40:24.835 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:40:34.836 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:40:44.836 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:40:54.837 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:41:04.838 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:41:09.775 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:41:09.776 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:41:09.776 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:41:14.839 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:41:24.840 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:41:34.840 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:41:44.841 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:41:54.841 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:42:04.842 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:42:09.777 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:42:09.778 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:42:09.778 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:42:14.843 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:42:24.844 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:42:34.844 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:42:44.846 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:42:54.847 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:43:04.848 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:43:09.779 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:43:09.779 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:43:09.780 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:43:14.848 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:43:24.849 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:43:34.849 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:43:44.850 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:43:54.851 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:44:04.851 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:44:09.781 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:44:09.781 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:44:09.782 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:44:14.852 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:44:24.853 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:44:34.853 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:44:44.855 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:44:54.855 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:45:04.857 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:45:09.783 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:45:09.784 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:45:09.784 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:45:14.857 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:45:24.858 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:45:34.859 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:45:44.859 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:45:54.860 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:46:04.860 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:46:09.785 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:46:09.786 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:46:09.786 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:46:14.861 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:46:24.862 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:46:34.863 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:46:44.864 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:46:54.864 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:47:04.866 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:47:09.787 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:47:09.788 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:47:09.788 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:47:14.867 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:47:24.867 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:47:34.868 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:47:44.869 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:47:54.869 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:48:04.870 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:48:09.789 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:48:09.790 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:48:09.791 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:48:14.871 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:48:24.872 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:48:34.872 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:48:44.873 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:48:54.874 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:49:04.874 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:49:09.792 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:49:09.793 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:49:09.793 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:49:14.875 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:49:24.876 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:49:34.877 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:49:44.877 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:49:54.878 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:50:04.879 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:50:09.795 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:50:09.795 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:50:09.796 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:50:14.880 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:50:24.881 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:50:34.881 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:50:44.882 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:50:54.883 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:51:04.884 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:51:09.797 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:51:09.797 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:51:09.798 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:51:14.884 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:51:24.885 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:51:34.886 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:51:44.887 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:51:54.887 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:52:04.888 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:52:09.799 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:52:09.800 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:52:09.800 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:52:14.890 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:52:24.890 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:52:34.891 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:52:44.891 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:52:54.892 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:53:04.893 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:53:09.801 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:53:09.802 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:53:09.802 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:53:14.893 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:53:24.894 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:53:34.895 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:53:44.895 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:53:54.896 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:54:04.897 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:54:09.803 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:54:09.804 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:54:09.805 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:54:14.898 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:54:24.898 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:54:34.899 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:54:44.900 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:54:54.900 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:55:04.901 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:55:09.805 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:55:09.806 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:55:09.807 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:55:14.902 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:55:24.903 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:55:34.903 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:55:44.904 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:55:54.904 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:56:04.905 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:56:09.808 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:56:09.809 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:56:09.810 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:56:14.906 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:56:24.907 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:56:34.907 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:56:44.908 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:56:54.909 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:57:04.910 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:57:09.811 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:57:09.812 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:57:09.812 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:57:14.911 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:57:24.912 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:57:34.913 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:57:44.913 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:57:54.914 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:58:04.915 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:58:09.813 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:58:09.814 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:58:09.814 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:58:14.915 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:58:24.916 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:58:34.917 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:58:44.917 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:58:54.918 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:59:04.918 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:59:09.815 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:59:09.816 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 08:59:09.817 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 08:59:14.919 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:59:24.920 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:59:34.920 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:59:44.921 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 08:59:54.921 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:00:04.923 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:00:09.818 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:00:09.819 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:00:09.819 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:00:14.924 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:00:24.925 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:00:34.926 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:00:44.926 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:00:54.927 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:01:04.928 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:01:09.820 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:01:09.821 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:01:09.821 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:01:14.930 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:01:24.930 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:01:34.932 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:01:44.933 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:01:54.934 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:02:04.936 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:02:09.823 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:02:09.823 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:02:09.824 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:02:14.936 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:02:24.937 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:02:34.937 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:02:44.938 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:02:54.939 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:03:04.940 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:03:09.824 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:03:09.825 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:03:09.825 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:03:14.941 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:03:24.942 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:03:34.942 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:03:44.943 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:03:54.944 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:04:04.945 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:04:09.826 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:04:09.827 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:04:09.828 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:04:14.946 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:04:24.949 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:04:34.949 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:04:44.950 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:04:54.951 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:05:04.952 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:05:09.830 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:05:09.831 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:05:09.831 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:05:14.952 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:05:24.953 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:05:34.954 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:05:44.954 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:05:54.955 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:06:04.956 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:06:09.832 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:06:09.832 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:06:09.833 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:06:14.956 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:06:24.957 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:06:34.958 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:06:44.959 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:06:54.960 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:07:04.961 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:07:09.834 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:07:09.835 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:07:09.835 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:07:14.962 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:07:24.963 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:07:34.964 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:07:44.964 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:07:54.965 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:08:04.966 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:08:09.836 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:08:09.836 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:08:09.837 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:08:14.967 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:08:24.968 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:08:34.968 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:08:44.969 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:08:54.970 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:09:04.970 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:09:09.838 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:09:09.839 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:09:09.839 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:09:14.971 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:09:24.972 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:09:34.972 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:09:44.973 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:09:54.974 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:10:04.974 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:10:09.840 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:10:09.841 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:10:09.841 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:10:14.975 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:10:24.976 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:10:34.977 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:10:44.978 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:10:54.978 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:11:04.979 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:11:09.842 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:11:09.843 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:11:09.844 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:11:14.980 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:11:24.981 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:11:34.982 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:11:44.982 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:11:54.983 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:12:04.984 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:12:09.845 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:12:09.845 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:12:09.846 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:12:14.984 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:12:24.985 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:12:34.986 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:12:44.987 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:12:54.987 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:13:04.988 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:13:09.847 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:13:09.847 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:13:09.848 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:13:14.989 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:13:24.989 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:13:34.990 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:13:44.990 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:13:54.991 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:14:04.992 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:14:09.849 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:14:09.849 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:14:09.850 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:14:14.993 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:14:24.994 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:14:34.994 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:14:44.995 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:14:54.995 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:15:04.996 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:15:09.851 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:15:09.851 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:15:09.852 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:15:14.997 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:15:24.997 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:15:34.998 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:15:44.999 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:15:55.000 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:16:05.001 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:16:09.853 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:16:09.853 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:16:09.854 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:16:15.001 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:16:25.002 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:16:35.003 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:16:45.004 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:16:55.005 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:17:05.006 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:17:09.855 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:17:09.855 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:17:09.856 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:17:15.006 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:17:25.007 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:17:35.008 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:17:45.009 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:17:55.009 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:18:05.010 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:18:09.857 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:18:09.858 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:18:09.858 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:18:15.011 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:18:25.012 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:18:35.012 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:18:45.013 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:18:55.014 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:19:05.015 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:19:09.859 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:19:09.860 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:19:09.860 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:19:15.015 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:19:25.016 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:19:35.017 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:19:45.018 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:19:55.019 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:20:05.022 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:20:09.861 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:20:09.862 980119 DEBUG oslo_service.periodic_task [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:20:09.862 980119 WARNING cinder.volume.manager [req-dc9d2b50-776b-4e2e-8595-af9341884303 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:20:15.023 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:20:25.024 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:20:35.025 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:20:45.025 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:20:55.026 980119 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:21:01.411 980023 INFO oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Caught SIGTERM, stopping children
>2018-06-25 09:21:01.413 980023 DEBUG oslo_concurrency.lockutils [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 09:21:01.414 980023 DEBUG oslo_concurrency.lockutils [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 09:21:01.414 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Stop services. stop /usr/lib/python2.7/site-packages/oslo_service/service.py:624
>2018-06-25 09:21:01.416 980023 DEBUG oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Killing children. stop /usr/lib/python2.7/site-packages/oslo_service/service.py:629
>2018-06-25 09:21:01.416 980023 INFO oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Waiting on 1 children to exit
>2018-06-25 09:21:04.857 980119 DEBUG oslo_concurrency.lockutils [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 09:21:04.858 980119 DEBUG oslo_concurrency.lockutils [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 09:21:09.903 980023 INFO oslo_service.service [req-16660ffc-594a-4a13-baa0-4e25fced4cd0 - - - - -] Child 980119 exited with status 0
>2018-06-25 09:21:15.047 298798 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
>2018-06-25 09:21:15.048 298798 DEBUG oslo_concurrency.lockutils [-] Acquired semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212
>2018-06-25 09:21:15.048 298798 DEBUG oslo_concurrency.lockutils [-] Releasing semaphore "singleton_lock" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
>2018-06-25 09:21:15.452 298798 DEBUG oslo_db.sqlalchemy.engines [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 09:21:15.614 298798 INFO cinder.rpc [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Automatically selected cinder-scheduler objects version 1.27 as minimum service version.
>2018-06-25 09:21:15.619 298798 INFO cinder.rpc [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Automatically selected cinder-scheduler RPC version 3.7 as minimum service version.
>2018-06-25 09:21:15.625 298798 DEBUG oslo_db.sqlalchemy.engines [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 09:21:15.632 298798 WARNING cinder.keymgr.conf_key_mgr [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] This key manager is insecure and is not recommended for production deployments
>2018-06-25 09:21:15.641 298798 INFO cinder.volume.manager [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Image-volume cache disabled for host hostgroup@tripleo_ceph.
>2018-06-25 09:21:15.646 298798 DEBUG oslo_db.sqlalchemy.engines [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 09:21:15.656 298798 INFO oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Starting 1 workers
>2018-06-25 09:21:15.659 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Started child 298965 _start_child /usr/lib/python2.7/site-packages/oslo_service/service.py:513
>2018-06-25 09:21:15.662 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Full set of CONF: wait /usr/lib/python2.7/site-packages/oslo_service/service.py:580
>2018-06-25 09:21:15.663 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2865
>2018-06-25 09:21:15.663 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Configuration options gathered from: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2866
>2018-06-25 09:21:15.663 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] command line args: ['--config-file', '/usr/share/cinder/cinder-dist.conf', '--config-file', '/etc/cinder/cinder.conf', '--logfile', '/var/log/cinder/volume.log'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2867
>2018-06-25 09:21:15.664 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] config files: ['/usr/share/cinder/cinder-dist.conf', '/etc/cinder/cinder.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2869
>2018-06-25 09:21:15.663 298965 INFO cinder.service [-] Starting cinder-volume node (version 11.1.0)
>2018-06-25 09:21:15.664 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] ================================================================================ log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2870
>2018-06-25 09:21:15.665 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] allow_availability_zone_fallback = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.665 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] allowed_direct_url_schemes     = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.666 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] api_rate_limit                 = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.666 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] auth_strategy                  = keystone log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.666 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_availability_zone      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.667 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_host                   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.667 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backup_api_class               = cinder.backup.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.667 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backup_manager                 = cinder.backup.manager.BackupManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.668 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backup_name_template           = backup-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.668 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backup_use_same_host           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.668 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backup_use_temp_snapshot       = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.669 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] chap_password                  = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.669 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] chap_username                  =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.670 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.670 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] cinder_internal_tenant_project_id = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.670 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] cinder_internal_tenant_user_id = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.671 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] cluster                        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.671 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] compute_api_class              = cinder.compute.nova.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.671 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] config_dir                     = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.672 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] config_file                    = ['/usr/share/cinder/cinder-dist.conf', '/etc/cinder/cinder.conf'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.672 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] consistencygroup_api_class     = cinder.consistencygroup.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.672 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] control_exchange               = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.673 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] db_driver                      = cinder.db log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.673 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] debug                          = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.673 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] default_availability_zone      = nova log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.674 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] default_group_type             = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.674 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'dogpile.core.dogpile=INFO'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.675 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] default_volume_type            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.675 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] driver_client_cert             = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.675 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] driver_client_cert_key         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.676 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] driver_data_namespace          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.676 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] driver_ssl_cert_path           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.676 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] driver_ssl_cert_verify         = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.677 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] driver_use_ssl                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.677 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] enable_new_services            = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.677 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] enable_unsupported_driver      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.678 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] enable_v1_api                  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.678 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] enable_v2_api                  = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.678 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] enable_v3_api                  = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.679 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] enabled_backends               = ['tripleo_ceph'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.679 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] enforce_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.679 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] fatal_exception_format_errors  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.680 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] filter_function                = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.680 298965 INFO cinder.volume.manager [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Starting volume driver RBDDriver (1.2.0)
>2018-06-25 09:21:15.680 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] glance_api_insecure            = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.681 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] glance_api_servers             = ['http://172.17.1.12:9292'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.681 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] glance_api_ssl_compression     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.681 298965 DEBUG cinder.volume.drivers.rbd [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:21:15.681 298798 WARNING oslo_config.cfg [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Option "glance_api_version" from group "DEFAULT" is deprecated for removal (Glance v1 support will be removed in Queens).  Its value may be silently ignored in the future.
>2018-06-25 09:21:15.682 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] glance_api_version             = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.682 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] glance_ca_certificates_file    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.682 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] glance_catalog_info            = image:glance:publicURL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.683 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.683 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] glance_num_retries             = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.683 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] glance_request_timeout         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.684 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] goodness_function              = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.684 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] graceful_shutdown_timeout      = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.684 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] group_api_class                = cinder.group.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.686 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] host                           = hostgroup log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.686 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iet_conf                       = /etc/iet/ietd.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.686 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] image_conversion_dir           = /var/lib/cinder/conversion log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.687 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] image_upload_use_cinder_backend = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.687 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] image_upload_use_internal_tenant = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.688 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] image_volume_cache_enabled     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.688 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] image_volume_cache_max_count   = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.688 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] image_volume_cache_max_size_gb = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.689 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] instance_format                = [instance: %(uuid)s]  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.689 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] instance_uuid_format           = [instance: %(uuid)s]  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.689 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iscsi_helper                   = lioadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.690 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iscsi_iotype                   = fileio log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.690 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iscsi_ip_address               = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.690 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iscsi_port                     = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.691 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iscsi_protocol                 = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.691 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iscsi_secondary_ip_addresses   = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.691 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iscsi_target_flags             =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.692 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iscsi_target_prefix            = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.692 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iscsi_write_cache              = on log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.693 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iser_helper                    = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.693 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iser_ip_address                = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.693 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iser_port                      = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.694 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] iser_target_prefix             = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.694 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] log_config_append              = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.694 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] log_date_format                = %Y-%m-%d %H:%M:%S log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.695 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] log_dir                        = /var/log/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.695 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] log_file                       = /var/log/cinder/volume.log log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.695 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] log_options                    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.696 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] logging_context_format_string  = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.696 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] logging_debug_format_suffix    = %(funcName)s %(pathname)s:%(lineno)d log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.696 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] logging_default_format_string  = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.697 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] logging_exception_prefix       = %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.697 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] logging_user_identity_format   = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.697 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] max_age                        = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.698 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] max_over_subscription_ratio    = 20.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.698 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] message_reap_interval          = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.698 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] message_ttl                    = 2592000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.699 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] migration_create_volume_timeout_secs = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.699 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] monkey_patch                   = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.699 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] monkey_patch_modules           = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.700 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] my_ip                          = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.700 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] no_snapshot_gb_quota           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.700 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] num_iser_scan_tries            = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.701 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] num_shell_tries                = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.701 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] num_volume_device_scan_tries   = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.701 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] os_privileged_user_auth_url    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.702 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] os_privileged_user_name        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.702 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] os_privileged_user_password    = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.702 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] os_privileged_user_tenant      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.703 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] osapi_max_limit                = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.703 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] osapi_volume_base_URL          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.703 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] osapi_volume_ext_list          = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.704 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.704 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] osapi_volume_listen            = controller-0.internalapi.localdomain log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.704 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] osapi_volume_listen_port       = 8776 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.705 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] osapi_volume_use_ssl           = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.705 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] osapi_volume_workers           = 4 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.705 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] per_volume_size_limit          = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.706 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] periodic_fuzzy_delay           = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.706 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] periodic_interval              = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.706 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] public_endpoint                = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.706 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] publish_errors                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.707 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] query_volume_filters           = ['name', 'status', 'metadata', 'availability_zone', 'bootable', 'group_id'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.707 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] quota_backup_gigabytes         = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.708 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] quota_backups                  = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.708 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] quota_consistencygroups        = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.708 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] quota_driver                   = cinder.quota.DbQuotaDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.709 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] quota_gigabytes                = 1000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.709 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] quota_groups                   = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.709 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] quota_snapshots                = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.710 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] quota_volumes                  = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.710 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] rate_limit_burst               = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.710 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] rate_limit_except_level        = CRITICAL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.711 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] rate_limit_interval            = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.711 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] replication_device             = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.711 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] report_discard_supported       = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.712 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] report_interval                = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.712 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] reservation_clean_interval     = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.712 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] reservation_expire             = 86400 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.713 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] reserved_percentage            = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.713 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] resource_query_filters_file    = /etc/cinder/resource_filters.json log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.713 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] rootwrap_config                = /etc/cinder/rootwrap.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.714 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] rpc_backend                    = rabbit log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.714 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] rpc_response_timeout           = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.714 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] run_external_periodic_tasks    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.715 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] scheduler_manager              = cinder.scheduler.manager.SchedulerManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.715 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] scst_target_driver             = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.715 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] scst_target_iqn_name           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.716 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] service_down_time              = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.716 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] snapshot_name_template         = snapshot-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.716 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] state_path                     = /var/lib/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.716 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] storage_availability_zone      = nova log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.717 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] storage_protocol               = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.717 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] syslog_log_facility            = LOG_USER log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.717 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] trace_flags                    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.718 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] transfer_api_class             = cinder.transfer.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.718 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] transport_url                  = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.717 298965 ERROR cinder.volume.drivers.rbd [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:21:15.717 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:21:15.717 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:21:15.717 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:21:15.717 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:21:15.717 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:21:15.717 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:21:15.719 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] until_refresh                  = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.719 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] use_chap_auth                  = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.719 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] use_default_quota_class        = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.719 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] use_journal                    = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.720 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] use_multipath_for_image_xfer   = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.720 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] use_stderr                     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.720 298965 DEBUG cinder.utils [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:21:15.720 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] use_syslog                     = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.721 298965 DEBUG cinder.utils [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:21:15.721 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_api_class               = cinder.volume.api.API log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.721 298965 DEBUG cinder.utils [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 09:21:15.721 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_backend_name            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.721 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_clear                   = zero log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.722 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_clear_ionice            = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.722 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_clear_size              = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.723 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_copy_blkio_cgroup_name  = cinder-volume-copy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.723 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_copy_bps_limit          = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.723 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_dd_blocksize            = 1M log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.724 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_manager                 = cinder.volume.manager.VolumeManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.724 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_name_template           = volume-%s log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.724 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_service_inithost_offload = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.725 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volume_usage_audit_period      = month log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.725 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] volumes_dir                    = /etc/cinder/volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.725 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] watch_log_file                 = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.726 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] zoning_mode                    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2879
>2018-06-25 09:21:15.726 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.amqp_auto_delete = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.727 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.amqp_durable_queues = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.727 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.conn_pool_min_size = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.727 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.conn_pool_ttl = 1200 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.728 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.fake_rabbit = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.728 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.heartbeat_rate = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.728 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.729 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.kombu_compression = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.729 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.kombu_failover_strategy = round-robin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.729 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.730 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.730 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_ha_queues = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.731 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_host = localhost log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.731 298798 WARNING oslo_config.cfg [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 09:21:15.731 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_hosts = ['localhost:5672'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.732 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_interval_max = 30 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.732 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.733 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_max_retries = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.733 298798 WARNING oslo_config.cfg [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 09:21:15.733 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.733 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_port = 5672 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.734 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.734 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_retry_backoff = 2 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.734 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_retry_interval = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.735 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.735 298798 WARNING oslo_config.cfg [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 09:21:15.735 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_userid = guest log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.736 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rabbit_virtual_host = / log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.736 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.rpc_conn_pool_size = 30 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.737 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.ssl      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.737 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.ssl_ca_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.737 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.ssl_cert_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.738 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.ssl_key_file =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.738 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_rabbit.ssl_version =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.738 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] coordination.backend_url       = file:///var/lib/cinder log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.739 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] coordination.heartbeat         = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.739 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] coordination.initial_reconnect_backoff = 0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.739 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] coordination.max_reconnect_backoff = 60.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.740 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.backend               = sqlalchemy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.740 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.connection            = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.741 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.connection_debug      = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.741 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.connection_trace      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.741 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.db_inc_retry_interval = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.742 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.db_max_retries        = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.742 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.db_max_retry_interval = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.742 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.db_retry_interval     = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.743 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.idle_timeout          = 3600 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.743 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.max_overflow          = 50 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.743 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.max_pool_size         = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.744 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.max_retries           = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.744 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.min_pool_size         = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.744 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.mysql_enable_ndb      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.745 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.mysql_sql_mode        = TRADITIONAL log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.745 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.pool_timeout          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.745 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.retry_interval        = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.746 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.slave_connection      = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.746 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.sqlite_synchronous    = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.746 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] database.use_db_reconnect      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.747 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_notifications.driver = ['messagingv2'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.747 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_notifications.retry = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.748 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_notifications.topics = ['notifications'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.748 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_messaging_notifications.transport_url = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.748 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_reports.file_event_handler = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.749 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_reports.file_event_handler_interval = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.749 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_reports.log_dir           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.749 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.backend_availability_zone = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.750 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.backend_host      = hostgroup log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.750 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.backup_use_temp_snapshot = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.750 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.chap_password     = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.751 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.chap_username     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.751 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.chiscsi_conf      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.751 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.driver_client_cert = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.752 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.driver_client_cert_key = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.752 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.driver_data_namespace = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.752 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.driver_ssl_cert_path = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.752 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.driver_ssl_cert_verify = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.753 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.driver_use_ssl    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.753 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.enable_unsupported_driver = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.753 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.enforce_multipath_for_image_xfer = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.754 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.extra_capabilities = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.754 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.filter_function   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.754 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.goodness_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.755 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iet_conf          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.755 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.image_upload_use_cinder_backend = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.755 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.image_upload_use_internal_tenant = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.756 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.image_volume_cache_enabled = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.756 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.image_volume_cache_max_count = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.756 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.image_volume_cache_max_size_gb = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.757 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iscsi_helper      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.757 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iscsi_iotype      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.757 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iscsi_ip_address  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.758 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iscsi_port        = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.758 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iscsi_protocol    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.758 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iscsi_secondary_ip_addresses = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.759 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iscsi_target_flags = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.759 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iscsi_target_prefix = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.759 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iscsi_write_cache = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.759 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iser_helper       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.760 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iser_ip_address   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.760 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iser_port         = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.760 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.iser_target_prefix = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.761 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.max_over_subscription_ratio = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.761 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.num_iser_scan_tries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.761 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.num_shell_tries   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.762 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.num_volume_device_scan_tries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.762 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rados_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.762 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rados_connection_interval = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.763 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rados_connection_retries = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.763 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rbd_ceph_conf     = /etc/ceph/ceph.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.763 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rbd_cluster_name  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.764 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rbd_flatten_volume_from_snapshot = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.764 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rbd_keyring_conf  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.764 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rbd_max_clone_depth = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.765 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rbd_pool          = volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.765 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rbd_secret_uuid   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.765 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rbd_store_chunk_size = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.766 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.rbd_user          = openstack log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.766 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.replication_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.766 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.replication_device = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.767 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.report_discard_supported = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.767 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.report_dynamic_total_capacity = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.767 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.reserved_percentage = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.767 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.scst_target_driver = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.768 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.scst_target_iqn_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.768 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.storage_protocol  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.768 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.suppress_requests_ssl_warnings = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.769 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.trace_flags       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.769 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.use_chap_auth     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.769 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.use_multipath_for_image_xfer = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.770 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.volume_backend_name = tripleo_ceph log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.770 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.volume_clear      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.770 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.volume_clear_ionice = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.771 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.volume_clear_size = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.771 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.volume_copy_blkio_cgroup_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.771 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.volume_copy_bps_limit = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.772 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.volume_dd_blocksize = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.772 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.volume_driver     = cinder.volume.drivers.rbd.RBDDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.772 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] tripleo_ceph.volumes_dir       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.773 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] profiler.connection_string     = messaging:// log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.773 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] profiler.enabled               = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.773 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] profiler.es_doc_type           = notification log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.774 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] profiler.es_scroll_size        = 10000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.774 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] profiler.es_scroll_time        = 2m log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.774 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] profiler.hmac_keys             = SECRET_KEY log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.775 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] profiler.sentinel_service_name = mymaster log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.775 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] profiler.socket_timeout        = 0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.775 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] profiler.trace_sqlalchemy      = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.776 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] privsep_osbrick.capabilities   = [21] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.776 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] privsep_osbrick.group          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.777 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] privsep_osbrick.helper_command = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.777 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] privsep_osbrick.user           = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.777 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.backend_availability_zone = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.777 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.backup_use_temp_snapshot = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.778 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.chap_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.778 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.chap_username =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.779 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.779 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.driver_client_cert = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.779 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.driver_client_cert_key = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.780 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.driver_data_namespace = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.780 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.driver_ssl_cert_path = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.780 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.driver_ssl_cert_verify = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.781 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.driver_use_ssl = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.781 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.enable_unsupported_driver = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.781 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.enforce_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.782 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.extra_capabilities = {} log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.782 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.filter_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.782 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.goodness_function = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.783 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iet_conf      = /etc/iet/ietd.conf log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.783 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.image_upload_use_cinder_backend = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.783 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.image_upload_use_internal_tenant = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.784 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.image_volume_cache_enabled = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.784 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.image_volume_cache_max_count = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.784 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.image_volume_cache_max_size_gb = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.785 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iscsi_helper  = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.785 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iscsi_iotype  = fileio log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.785 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iscsi_ip_address = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.786 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iscsi_port    = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.786 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iscsi_protocol = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.786 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iscsi_secondary_ip_addresses = [] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.787 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iscsi_target_flags =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.787 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iscsi_target_prefix = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.787 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iscsi_write_cache = on log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.788 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iser_helper   = tgtadm log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.788 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iser_ip_address = 10.0.0.101 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.788 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iser_port     = 3260 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.789 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.789 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.max_over_subscription_ratio = 20.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.789 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.num_iser_scan_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.790 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.num_shell_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.790 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.num_volume_device_scan_tries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.790 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rados_connect_timeout = -1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.791 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rados_connection_interval = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.791 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rados_connection_retries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.791 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rbd_ceph_conf =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.792 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rbd_cluster_name = ceph log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.792 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rbd_flatten_volume_from_snapshot = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.792 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rbd_keyring_conf =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.793 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rbd_max_clone_depth = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.793 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rbd_pool      = rbd log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.793 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rbd_secret_uuid = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.794 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rbd_store_chunk_size = 4 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.794 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.rbd_user      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.794 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.replication_connect_timeout = 5 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.795 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.replication_device = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.795 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.report_discard_supported = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.795 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.report_dynamic_total_capacity = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.796 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.reserved_percentage = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.796 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.scst_target_driver = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.796 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.scst_target_iqn_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.797 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.storage_protocol = iscsi log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.797 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.suppress_requests_ssl_warnings = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.797 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.trace_flags   = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.798 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.use_chap_auth = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.798 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.use_multipath_for_image_xfer = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.798 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.volume_backend_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.799 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.volume_clear  = zero log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.799 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.volume_clear_ionice = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.799 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.volume_clear_size = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.800 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.800 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.volume_copy_bps_limit = 0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.800 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.volume_dd_blocksize = 1M log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.801 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.801 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] backend_defaults.volumes_dir   = /var/lib/cinder/volumes log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.802 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_concurrency.disable_process_locking = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.802 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_concurrency.lock_path     = /var/lib/cinder/tmp log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.802 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] barbican.auth_endpoint         = http://localhost/identity/v3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.803 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] barbican.barbican_api_version  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.803 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] barbican.barbican_endpoint     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.803 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] barbican.number_of_retries     = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.804 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] barbican.retry_delay           = 1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.804 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] barbican.verify_ssl            = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.805 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.admin_password = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.805 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.admin_tenant_name = admin log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.805 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.admin_token = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.806 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.admin_user  = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.806 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.auth_admin_prefix =  log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.806 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.auth_host   = 127.0.0.1 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.807 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.auth_port   = 35357 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.807 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.auth_protocol = http log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.807 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.auth_section = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.808 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.auth_type   = password log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.808 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.auth_uri    = http://172.17.1.12:5000 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.808 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.auth_version = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.809 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.cache       = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.809 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.cafile      = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.810 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.certfile    = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.810 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.check_revocations_for_cached = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.810 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.delay_auth_decision = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.811 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.enforce_token_bind = permissive log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.811 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.hash_algorithms = ['md5'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.811 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.http_connect_timeout = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.812 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.http_request_max_retries = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.812 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.identity_uri = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.812 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.include_service_catalog = True log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.813 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.insecure    = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.813 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.keyfile     = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.813 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.memcache_pool_conn_get_timeout = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.814 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.memcache_pool_dead_retry = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.814 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.memcache_pool_maxsize = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.814 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.memcache_pool_socket_timeout = 3 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.815 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.memcache_pool_unused_timeout = 60 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.815 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.memcache_secret_key = **** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.815 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.memcache_security_strategy = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.816 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.memcache_use_advanced_pool = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.816 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.memcached_servers = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.816 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.region_name = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.817 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.revocation_cache_time = 10 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.817 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.service_token_roles = ['service'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.817 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.service_token_roles_required = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.818 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.signing_dir = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.818 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] keystone_authtoken.token_cache_time = 300 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.818 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_versionedobjects.fatal_exception_format_errors = False log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.819 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_policy.policy_default_rule = default log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.819 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_policy.policy_dirs        = ['policy.d'] log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.819 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] oslo_policy.policy_file        = /etc/cinder/policy.json log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.820 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] key_manager.api_class          = cinder.keymgr.conf_key_mgr.ConfKeyManager log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.820 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] key_manager.fixed_key          = None log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2887
>2018-06-25 09:21:15.820 298798 DEBUG oslo_service.service [req-ac198795-d150-4253-a1e1-99d274f81296 - - - - -] ******************************************************************************** log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2889
>2018-06-25 09:21:25.722 298965 DEBUG cinder.volume.drivers.rbd [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:21:25.759 298965 ERROR cinder.volume.drivers.rbd [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:21:25.759 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:21:25.759 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:21:25.759 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:21:25.759 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:21:25.759 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:21:25.759 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:21:25.761 298965 DEBUG cinder.utils [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:21:25.762 298965 DEBUG cinder.utils [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Have been at this for 10.081 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:21:25.762 298965 DEBUG cinder.utils [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 09:21:45.762 298965 DEBUG cinder.volume.drivers.rbd [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:21:45.799 298965 ERROR cinder.volume.drivers.rbd [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:21:45.799 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:21:45.799 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:21:45.799 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:21:45.799 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:21:45.799 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:21:45.799 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:21:45.801 298965 DEBUG cinder.utils [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:21:45.801 298965 DEBUG cinder.utils [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Have been at this for 30.121 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Failed to initialize driver.: VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 431, in init_host
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager     self.driver.check_for_setup_error()
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 272, in check_for_setup_error
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager     with RADOSClient(self):
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 154, in __init__
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager     self.cluster, self.ioctx = driver._connect_to_rados(pool)
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 330, in _connect_to_rados
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager     return _do_conn(pool, remote, timeout)
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 799, in _wrapper
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager     return r.call(f, *args, **kwargs)
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 229, in call
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager     raise attempt.get()
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager     six.reraise(self.value[0], self.value[1], self.value[2])
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 328, in _do_conn
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager     raise exception.VolumeBackendAPIException(data=msg)
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
>2018-06-25 09:21:45.802 298965 ERROR cinder.volume.manager 
>2018-06-25 09:21:45.804 298965 DEBUG cinder.service [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Creating RPC server for service cinder-volume start /usr/lib/python2.7/site-packages/cinder/service.py:219
>2018-06-25 09:21:45.813 298965 DEBUG oslo_db.sqlalchemy.engines [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 09:21:45.821 298965 DEBUG cinder.service [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Pinning object versions for RPC server serializer to 1.27 start /usr/lib/python2.7/site-packages/cinder/service.py:226
>2018-06-25 09:21:45.824 298965 WARNING oslo_config.cfg [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Option "rabbit_port" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 09:21:45.824 298965 WARNING oslo_config.cfg [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Option "rabbit_userid" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 09:21:45.824 298965 WARNING oslo_config.cfg [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Option "rabbit_password" from group "oslo_messaging_rabbit" is deprecated for removal (Replaced by [DEFAULT]/transport_url).  Its value may be silently ignored in the future.
>2018-06-25 09:21:45.949 298965 INFO cinder.volume.manager [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Initializing RPC dependent components of volume driver RBDDriver (1.2.0)
>2018-06-25 09:21:45.949 298965 ERROR cinder.utils [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Volume driver RBDDriver not initialized
>2018-06-25 09:21:45.950 298965 ERROR cinder.volume.manager [req-a49c72d9-aa8f-4a45-9479-1d5b71265058 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:21:55.951 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:22:02.026 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume reschedule parameters: True retry: {u'num_attempts': 1, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1075
>2018-06-25 09:22:02.068 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (72d0ecae-3fdb-4c20-b930-f9c10f053c50) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-25 09:22:02.071 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f5d4027a-cf00-4146-b86f-52dc95f819ac) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.109 298965 DEBUG oslo_db.sqlalchemy.engines [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 09:22:02.139 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f5d4027a-cf00-4146-b86f-52dc95f819ac) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:01Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:01Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.142 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (da0213e2-ab48-4e4c-82f3-6e9948f555d9) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.144 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (da0213e2-ab48-4e4c-82f3-6e9948f555d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.146 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c8528e6-11b2-4374-9854-2fc98f610348) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.148 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c8528e6-11b2-4374-9854-2fc98f610348) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', 'type': 'raw', 'volume_id': 'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.150 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1e2c876a-b9b0-4121-ae82-5148545a23f8) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.201 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1e2c876a-b9b0-4121-ae82-5148545a23f8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.203 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0a44c108-cb56-4aa1-8590-1db65a66daf1) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.204 298965 ERROR cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Unable to create volume. Volume driver RBDDriver not initialized
>2018-06-25 09:22:02.211 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0a44c108-cb56-4aa1-8590-1db65a66daf1) transitioned into state 'FAILURE' from state 'RUNNING'
>5 predecessors (most recent first):
>  Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:01Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:01Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fb2b761e8d0>}, 'provides': None}
>  |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:01Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:01Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(ac65bfb6-fba7-4603-9c78-12d2a8c13ee5),volume_id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fb2b761e8d0>}, 'provides': {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', 'type': 'raw', 'volume_id': 'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5'}}
>     |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:01Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:01Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'95f34b3d-8ea2-4bb9-80b6-495e3478196e', u'3840a8f6-9e80-4c9d-8b18-ee25fa688295'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 2, u'display_name': u'sd'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'95f34b3d-8ea2-4bb9-80b6-495e3478196e', u'3840a8f6-9e80-4c9d-8b18-ee25fa688295'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 2, u'display_name': u'sd'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'size': 2, u'display_name': u'sd', u'display_description': None, u'cluster_name': None, u'name_id': u'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'volume_admin_metadata': [], u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'host': None, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'bootable': False, u'created_at': u'2018-06-25T09:22:01.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'group_id': None}, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 1, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph']}, u'metadata': {}, u'resource_type': {}, u'size': 2}, 'context': <cinder.context.RequestContext object at 0x7fb2b761e8d0>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(ac65bfb6-fba7-4603-9c78-12d2a8c13ee5),volume_id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None}
>        |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:01Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:01Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fb2b761e8d0>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:01Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:01Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)}
>           |__Flow 'volume_create_manager': DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:02.211 298965 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-25 09:22:02.211 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 09:22:02.211 298965 ERROR cinder.volume.manager     result = task.execute(**arguments)
>2018-06-25 09:22:02.211 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 925, in execute
>2018-06-25 09:22:02.211 298965 ERROR cinder.volume.manager     raise exception.DriverNotInitialized()
>2018-06-25 09:22:02.211 298965 ERROR cinder.volume.manager DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:02.211 298965 ERROR cinder.volume.manager 
>2018-06-25 09:22:02.215 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0a44c108-cb56-4aa1-8590-1db65a66daf1) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.217 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0a44c108-cb56-4aa1-8590-1db65a66daf1) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:02.218 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1e2c876a-b9b0-4121-ae82-5148545a23f8) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.220 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1e2c876a-b9b0-4121-ae82-5148545a23f8) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:02.222 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c8528e6-11b2-4374-9854-2fc98f610348) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.224 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c8528e6-11b2-4374-9854-2fc98f610348) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:02.225 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (da0213e2-ab48-4e4c-82f3-6e9948f555d9) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.226 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Updating volume ac65bfb6-fba7-4603-9c78-12d2a8c13ee5 with {'host': None, 'scheduled_at': datetime.datetime(2018, 6, 25, 9, 22, 2, 226702)}. _pre_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:121
>2018-06-25 09:22:02.233 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume ac65bfb6-fba7-4603-9c78-12d2a8c13ee5: re-scheduling SchedulerAPI.create_volume attempt 1 due to Volume driver not ready. _reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:148
>2018-06-25 09:22:02.238 298965 INFO cinder.rpc [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Automatically selected cinder-volume objects version 1.27 as minimum service version.
>2018-06-25 09:22:02.275 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume ac65bfb6-fba7-4603-9c78-12d2a8c13ee5: re-scheduled _post_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:160
>2018-06-25 09:22:02.276 298965 DEBUG cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:22:02.315 298965 ERROR cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:02.315 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:22:02.315 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:22:02.315 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:22:02.315 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:22:02.315 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:02.315 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:22:02.317 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:22:02.317 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:22:02.317 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 09:22:02.429 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume reschedule parameters: True retry: {u'num_attempts': 2, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1075
>2018-06-25 09:22:02.443 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (eb16a133-f7e0-4a6e-8a76-1684eebef1f0) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-25 09:22:02.446 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ca34088e-60b9-4e1f-842d-8b7788d271f8) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.464 298965 DEBUG oslo_db.sqlalchemy.engines [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 09:22:02.488 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ca34088e-60b9-4e1f-842d-8b7788d271f8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.491 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8bf2f5ca-eee0-44c6-af5f-9df9e3df8703) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.493 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8bf2f5ca-eee0-44c6-af5f-9df9e3df8703) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.495 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ae8785a0-2dbe-4ea5-838d-9f398918dd5a) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.497 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ae8785a0-2dbe-4ea5-838d-9f398918dd5a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', 'type': 'raw', 'volume_id': 'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.499 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7e7f8e4e-760d-4488-a0ad-60fd6beb327f) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.523 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7e7f8e4e-760d-4488-a0ad-60fd6beb327f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.524 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8a6a632f-6b35-4d6c-ad9b-17a086c3fdfd) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.525 298965 ERROR cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Unable to create volume. Volume driver RBDDriver not initialized
>2018-06-25 09:22:02.530 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8a6a632f-6b35-4d6c-ad9b-17a086c3fdfd) transitioned into state 'FAILURE' from state 'RUNNING'
>5 predecessors (most recent first):
>  Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fb2b735cc90>}, 'provides': None}
>  |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(ac65bfb6-fba7-4603-9c78-12d2a8c13ee5),volume_id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fb2b735cc90>}, 'provides': {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', 'type': 'raw', 'volume_id': 'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5'}}
>     |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'95f34b3d-8ea2-4bb9-80b6-495e3478196e', u'3840a8f6-9e80-4c9d-8b18-ee25fa688295'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 2, u'display_name': u'sd'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'95f34b3d-8ea2-4bb9-80b6-495e3478196e', u'3840a8f6-9e80-4c9d-8b18-ee25fa688295'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 2, u'display_name': u'sd'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'size': 2, u'display_name': u'sd', u'display_description': None, u'cluster_name': None, u'name_id': u'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'volume_admin_metadata': [], u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'admin_metadata': {}, u'host': None, u'glance_metadata': {}, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'bootable': False, u'created_at': u'2018-06-25T09:22:01.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'group_id': None}, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 2, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']}, u'metadata': {}, u'resource_type': {}, u'size': 2}, 'context': <cinder.context.RequestContext object at 0x7fb2b735cc90>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(ac65bfb6-fba7-4603-9c78-12d2a8c13ee5),volume_id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None}
>        |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fb2b735cc90>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)}
>           |__Flow 'volume_create_manager': DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:02.530 298965 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-25 09:22:02.530 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 09:22:02.530 298965 ERROR cinder.volume.manager     result = task.execute(**arguments)
>2018-06-25 09:22:02.530 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 925, in execute
>2018-06-25 09:22:02.530 298965 ERROR cinder.volume.manager     raise exception.DriverNotInitialized()
>2018-06-25 09:22:02.530 298965 ERROR cinder.volume.manager DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:02.530 298965 ERROR cinder.volume.manager 
>2018-06-25 09:22:02.535 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8a6a632f-6b35-4d6c-ad9b-17a086c3fdfd) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.536 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8a6a632f-6b35-4d6c-ad9b-17a086c3fdfd) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:02.538 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7e7f8e4e-760d-4488-a0ad-60fd6beb327f) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.540 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7e7f8e4e-760d-4488-a0ad-60fd6beb327f) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:02.542 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ae8785a0-2dbe-4ea5-838d-9f398918dd5a) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.543 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ae8785a0-2dbe-4ea5-838d-9f398918dd5a) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:02.545 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8bf2f5ca-eee0-44c6-af5f-9df9e3df8703) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.546 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Updating volume ac65bfb6-fba7-4603-9c78-12d2a8c13ee5 with {'host': None, 'scheduled_at': datetime.datetime(2018, 6, 25, 9, 22, 2, 546712)}. _pre_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:121
>2018-06-25 09:22:02.553 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume ac65bfb6-fba7-4603-9c78-12d2a8c13ee5: re-scheduling SchedulerAPI.create_volume attempt 2 due to Volume driver not ready. _reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:148
>2018-06-25 09:22:02.570 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume ac65bfb6-fba7-4603-9c78-12d2a8c13ee5: re-scheduled _post_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:160
>2018-06-25 09:22:02.570 298965 DEBUG cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:22:02.609 298965 ERROR cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:02.609 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:22:02.609 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:22:02.609 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:22:02.609 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:22:02.609 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:02.609 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:22:02.611 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:22:02.611 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 0.04 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:22:02.612 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 09:22:02.837 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume reschedule parameters: True retry: {u'num_attempts': 3, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']} get_flow /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:1075
>2018-06-25 09:22:02.853 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (015ccb80-ce2f-4a31-9e1c-de188cf6c29e) transitioned into state 'RUNNING' from state 'PENDING' _flow_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:145
>2018-06-25 09:22:02.856 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (865ec4a8-b911-4d4a-be41-f45667ac6f2d) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.877 298965 DEBUG oslo_db.sqlalchemy.engines [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:285
>2018-06-25 09:22:02.919 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (865ec4a8-b911-4d4a-be41-f45667ac6f2d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.924 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (246c6531-cbf4-4cb0-800d-b57710b9236b) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.926 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (246c6531-cbf4-4cb0-800d-b57710b9236b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.928 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dc59382d-4d5f-4ce8-94a9-4039727a3caa) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.930 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dc59382d-4d5f-4ce8-94a9-4039727a3caa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', 'type': 'raw', 'volume_id': 'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5'}' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.931 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (14f6123e-d848-453e-90da-d1ccf2d42aad) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.954 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (14f6123e-d848-453e-90da-d1ccf2d42aad) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:183
>2018-06-25 09:22:02.956 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5db30bcb-83fd-45fb-8613-67da43a5160f) transitioned into state 'RUNNING' from state 'PENDING' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.957 298965 ERROR cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Unable to create volume. Volume driver RBDDriver not initialized
>2018-06-25 09:22:02.965 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5db30bcb-83fd-45fb-8613-67da43a5160f) transitioned into state 'FAILURE' from state 'RUNNING'
>5 predecessors (most recent first):
>  Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fb2b738e2d0>}, 'provides': None}
>  |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(ac65bfb6-fba7-4603-9c78-12d2a8c13ee5),volume_id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,volume_properties=VolumeProperties,volume_type=None), 'context': <cinder.context.RequestContext object at 0x7fb2b738e2d0>}, 'provides': {'status': u'creating', 'volume_size': 2, 'volume_name': 'volume-ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', 'type': 'raw', 'volume_id': 'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5'}}
>     |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'filter_properties': {u'config_options': {}, u'request_spec': {u'resource_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'95f34b3d-8ea2-4bb9-80b6-495e3478196e', u'3840a8f6-9e80-4c9d-8b18-ee25fa688295'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 2, u'display_name': u'sd'}, u'source_replicaid': None, u'volume_properties': {u'status': u'creating', u'volume_type_id': None, u'group_id': None, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'reservations': [u'95f34b3d-8ea2-4bb9-80b6-495e3478196e', u'3840a8f6-9e80-4c9d-8b18-ee25fa688295'], u'multiattach': False, u'attach_status': u'detached', u'source_volid': None, u'cgsnapshot_id': None, u'metadata': {}, u'qos_specs': None, u'encryption_key_id': None, u'display_description': None, u'snapshot_id': None, u'source_replicaid': None, u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'consistencygroup_id': None, u'size': 2, u'display_name': u'sd'}, u'source_volid': None, u'cgsnapshot_id': None, u'volume': {u'migration_status': None, u'provider_id': None, u'availability_zone': u'nova', u'terminated_at': None, u'updated_at': None, u'provider_geometry': None, u'replication_extended_status': None, u'replication_status': None, u'snapshot_id': None, u'ec2_id': None, u'deleted_at': None, u'id': u'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'size': 2, u'display_name': u'sd', u'display_description': None, u'cluster_name': None, u'name_id': u'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'volume_admin_metadata': [], u'project_id': u'678e5c6dd60648fe9047211c112d8c2f', u'launched_at': None, u'scheduled_at': None, u'status': u'creating', u'volume_type_id': None, u'multiattach': False, u'deleted': False, u'provider_location': None, u'volume_glance_metadata': [], u'admin_metadata': {}, u'host': None, u'glance_metadata': {}, u'consistencygroup_id': None, u'source_volid': None, u'provider_auth': None, u'previous_status': None, u'group_id': None, u'name': u'volume-ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'bootable': False, u'created_at': u'2018-06-25T09:22:01.000000', u'attach_status': u'detached', u'volume_metadata': [], u'_name_id': None, u'encryption_key_id': None, u'replication_driver_data': None, u'metadata': {}}, u'image_id': None, u'snapshot_id': None, u'consistencygroup_id': None, u'volume_type': None, u'volume_id': u'ac65bfb6-fba7-4603-9c78-12d2a8c13ee5', u'group_id': None}, u'user_id': u'ff3f45338f6c4c51aba850938fc58b2e', u'availability_zone': u'nova', u'volume_type': None, u'qos_specs': None, u'retry': {u'num_attempts': 3, u'backends': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph'], u'hosts': [u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph', u'hostgroup@tripleo_ceph#tripleo_ceph']}, u'metadata': {}, u'resource_type': {}, u'size': 2}, 'context': <cinder.context.RequestContext object at 0x7fb2b738e2d0>, 'request_spec': RequestSpec(CG_backend=<?>,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=None,snapshot_id=None,source_replicaid=None,source_volid=None,volume=Volume(ac65bfb6-fba7-4603-9c78-12d2a8c13ee5),volume_id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,volume_properties=VolumeProperties,volume_type=None)}, 'provides': None}
>        |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'volume': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None), 'context': <cinder.context.RequestContext object at 0x7fb2b738e2d0>}, 'provides': Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2018-06-25T09:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='sd',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=ac65bfb6-fba7-4603-9c78-12d2a8c13ee5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='678e5c6dd60648fe9047211c112d8c2f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2018-06-25T09:22:02Z,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2018-06-25T09:22:02Z,user_id='ff3f45338f6c4c51aba850938fc58b2e',volume_attachment=<?>,volume_type=<?>,volume_type_id=None)}
>           |__Flow 'volume_create_manager': DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:02.965 298965 ERROR cinder.volume.manager Traceback (most recent call last):
>2018-06-25 09:22:02.965 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 09:22:02.965 298965 ERROR cinder.volume.manager     result = task.execute(**arguments)
>2018-06-25 09:22:02.965 298965 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 925, in execute
>2018-06-25 09:22:02.965 298965 ERROR cinder.volume.manager     raise exception.DriverNotInitialized()
>2018-06-25 09:22:02.965 298965 ERROR cinder.volume.manager DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:02.965 298965 ERROR cinder.volume.manager 
>2018-06-25 09:22:02.971 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5db30bcb-83fd-45fb-8613-67da43a5160f) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.973 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5db30bcb-83fd-45fb-8613-67da43a5160f) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:02.976 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (14f6123e-d848-453e-90da-d1ccf2d42aad) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.978 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (14f6123e-d848-453e-90da-d1ccf2d42aad) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:02.979 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dc59382d-4d5f-4ce8-94a9-4039727a3caa) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.981 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dc59382d-4d5f-4ce8-94a9-4039727a3caa) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:02.983 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (246c6531-cbf4-4cb0-800d-b57710b9236b) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:02.984 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Updating volume ac65bfb6-fba7-4603-9c78-12d2a8c13ee5 with {'host': None, 'scheduled_at': datetime.datetime(2018, 6, 25, 9, 22, 2, 984100)}. _pre_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:121
>2018-06-25 09:22:02.990 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume ac65bfb6-fba7-4603-9c78-12d2a8c13ee5: re-scheduling SchedulerAPI.create_volume attempt 3 due to Volume driver not ready. _reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:148
>2018-06-25 09:22:03.004 298965 DEBUG cinder.volume.flows.manager.create_volume [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Volume ac65bfb6-fba7-4603-9c78-12d2a8c13ee5: re-scheduled _post_reschedule /usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py:160
>2018-06-25 09:22:03.005 298965 DEBUG cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:22:03.038 298965 ERROR cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:03.038 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:22:03.038 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:22:03.038 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:22:03.038 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:22:03.038 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:03.038 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:22:03.040 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 1 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:22:03.040 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 0.035 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:22:03.040 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 10.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 09:22:05.952 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:22:12.318 298965 DEBUG cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:22:12.350 298965 ERROR cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:12.350 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:22:12.350 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:22:12.350 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:22:12.350 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:22:12.350 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:12.350 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:22:12.352 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:22:12.352 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 10.075 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:22:12.352 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 09:22:12.612 298965 DEBUG cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:22:12.646 298965 ERROR cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:12.646 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:22:12.646 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:22:12.646 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:22:12.646 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:22:12.646 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:12.646 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:22:12.648 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:22:12.648 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 10.077 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:22:12.649 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 09:22:13.041 298965 DEBUG cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:22:13.074 298965 ERROR cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:13.074 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:22:13.074 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:22:13.074 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:22:13.074 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:22:13.074 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:13.074 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:22:13.077 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 2 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:22:13.078 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 10.073 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:22:13.078 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Sleeping for 20.0 seconds _backoff_sleep /usr/lib/python2.7/site-packages/cinder/utils.py:777
>2018-06-25 09:22:15.953 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:22:25.954 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:22:32.353 298965 DEBUG cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:22:32.388 298965 ERROR cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:32.388 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:22:32.388 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:22:32.388 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:22:32.388 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:22:32.388 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:32.388 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:22:32.390 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:22:32.390 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:22:32.393 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (da0213e2-ab48-4e4c-82f3-6e9948f555d9) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
>2018-06-25 09:22:32.395 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f5d4027a-cf00-4146-b86f-52dc95f819ac) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:32.397 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f5d4027a-cf00-4146-b86f-52dc95f819ac) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:32.399 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (72d0ecae-3fdb-4c20-b930-f9c10f053c50) transitioned into state 'REVERTED' from state 'RUNNING'
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Exception during message handling: DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "<string>", line 2, in create_volume
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     _run_flow()
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     flow_engine.run()
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     failures[0].reraise()
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     six.reraise(*self._exc_info)
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 925, in execute
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server     raise exception.DriverNotInitialized()
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:32.400 298965 ERROR oslo_messaging.rpc.server 
>2018-06-25 09:22:32.649 298965 DEBUG cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:22:32.688 298965 ERROR cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:32.688 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:22:32.688 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:22:32.688 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:22:32.688 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:22:32.688 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:32.688 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:22:32.690 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:22:32.690 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 30.119 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:22:32.692 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8bf2f5ca-eee0-44c6-af5f-9df9e3df8703) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
>2018-06-25 09:22:32.694 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ca34088e-60b9-4e1f-842d-8b7788d271f8) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:32.696 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ca34088e-60b9-4e1f-842d-8b7788d271f8) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:32.698 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (eb16a133-f7e0-4a6e-8a76-1684eebef1f0) transitioned into state 'REVERTED' from state 'RUNNING'
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Exception during message handling: DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "<string>", line 2, in create_volume
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     _run_flow()
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     flow_engine.run()
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     failures[0].reraise()
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     six.reraise(*self._exc_info)
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 925, in execute
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server     raise exception.DriverNotInitialized()
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:32.699 298965 ERROR oslo_messaging.rpc.server 
>2018-06-25 09:22:33.079 298965 DEBUG cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:308
>2018-06-25 09:22:33.115 298965 ERROR cinder.volume.drivers.rbd [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Error connecting to ceph cluster.: Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:33.115 298965 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):
>2018-06-25 09:22:33.115 298965 ERROR cinder.volume.drivers.rbd   File "/usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py", line 321, in _do_conn
>2018-06-25 09:22:33.115 298965 ERROR cinder.volume.drivers.rbd     client.connect()
>2018-06-25 09:22:33.115 298965 ERROR cinder.volume.drivers.rbd   File "rados.pyx", line 785, in rados.Rados.connect (rados.c:8969)
>2018-06-25 09:22:33.115 298965 ERROR cinder.volume.drivers.rbd Error: error connecting to the cluster: error code 95
>2018-06-25 09:22:33.115 298965 ERROR cinder.volume.drivers.rbd 
>2018-06-25 09:22:33.117 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Failed attempt 3 _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:783
>2018-06-25 09:22:33.118 298965 DEBUG cinder.utils [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Have been at this for 30.113 seconds _print_stop /usr/lib/python2.7/site-packages/cinder/utils.py:785
>2018-06-25 09:22:33.119 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (246c6531-cbf4-4cb0-800d-b57710b9236b) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
>2018-06-25 09:22:33.121 298965 DEBUG cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (865ec4a8-b911-4d4a-be41-f45667ac6f2d) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python2.7/site-packages/taskflow/listeners/logging.py:194
>2018-06-25 09:22:33.122 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (865ec4a8-b911-4d4a-be41-f45667ac6f2d) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
>2018-06-25 09:22:33.125 298965 WARNING cinder.volume.manager [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Flow 'volume_create_manager' (015ccb80-ce2f-4a31-9e1c-de188cf6c29e) transitioned into state 'REVERTED' from state 'RUNNING'
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server [req-335d2f32-9d46-49e7-9c90-84f3b0a391c0 ff3f45338f6c4c51aba850938fc58b2e 678e5c6dd60648fe9047211c112d8c2f - default default] Exception during message handling: DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "<string>", line 2, in create_volume
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/objects/cleanable.py", line 207, in wrapper
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 652, in create_volume
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     _run_flow()
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 644, in _run_flow
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     flow_engine.run()
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 247, in run
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 336, in reraise_if_any
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     failures[0].reraise()
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/types/failure.py", line 343, in reraise
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     six.reraise(*self._exc_info)
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/manager/create_volume.py", line 925, in execute
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server     raise exception.DriverNotInitialized()
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server DriverNotInitialized: Volume driver not ready.
>2018-06-25 09:22:33.125 298965 ERROR oslo_messaging.rpc.server 
>2018-06-25 09:22:35.954 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:22:37.951 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:22:37.952 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:22:37.952 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:22:45.955 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:22:55.955 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:23:05.957 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:23:15.958 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:23:25.959 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:23:35.960 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:23:37.954 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:23:37.954 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:23:37.955 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:23:45.961 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:23:55.961 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:24:05.963 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:24:15.964 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:24:25.965 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:24:35.965 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:24:37.956 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:24:37.957 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:24:37.957 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:24:45.966 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:24:55.967 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:25:05.968 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:25:15.969 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:25:25.970 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:25:35.970 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:25:37.958 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:25:37.959 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:25:37.960 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:25:45.971 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:25:55.972 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:26:05.973 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:26:15.973 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:26:25.975 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:26:35.976 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:26:37.961 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:26:37.961 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:26:37.962 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:26:45.977 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:26:55.978 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:27:05.980 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:27:15.980 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:27:25.981 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:27:35.982 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:27:37.963 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:27:37.964 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:27:37.964 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:27:45.984 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:27:55.984 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:28:05.986 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:28:15.987 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:28:25.987 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:28:35.988 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:28:37.965 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:28:37.966 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:28:37.967 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:28:45.989 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:28:55.990 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:29:05.990 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:29:15.991 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:29:25.991 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:29:35.992 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:29:37.968 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:29:37.968 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:29:37.969 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:29:45.993 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:29:55.994 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:30:05.995 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:30:15.996 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:30:25.997 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:30:35.997 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:30:37.970 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:30:37.971 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:30:37.971 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:30:45.998 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:30:55.998 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:31:05.999 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:31:16.000 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:31:26.001 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:31:36.001 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:31:37.972 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:31:37.973 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:31:37.974 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:31:46.002 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:31:56.003 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:32:06.004 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:32:16.004 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:32:26.005 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:32:36.006 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:32:37.975 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:32:37.975 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:32:37.975 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:32:46.007 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:32:56.007 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:33:06.009 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:33:16.010 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:33:26.011 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:33:36.011 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:33:37.976 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:33:37.977 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:33:37.977 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:33:46.012 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:33:56.013 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:34:06.014 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:34:16.014 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:34:26.015 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:34:36.016 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:34:37.978 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:34:37.979 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:34:37.979 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:34:46.017 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:34:56.018 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:35:06.019 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:35:16.020 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:35:26.020 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:35:36.021 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:35:37.980 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:35:37.981 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:35:37.981 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:35:46.022 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:35:56.022 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:36:06.023 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:36:16.024 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:36:26.025 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:36:36.026 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:36:37.982 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:36:37.983 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:36:37.983 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:36:46.027 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:36:56.027 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:37:06.028 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:37:16.029 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:37:26.030 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:37:36.030 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:37:37.984 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:37:37.985 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:37:37.986 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:37:46.031 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:37:56.032 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:38:06.033 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:38:16.034 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:38:26.034 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:38:36.035 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:38:37.987 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:38:37.987 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:38:37.988 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:38:46.035 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:38:56.036 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:39:06.037 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:39:16.038 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:39:26.038 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:39:36.039 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:39:37.988 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:39:37.989 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:39:37.989 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:39:46.040 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:39:56.040 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:40:06.041 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:40:16.042 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:40:26.043 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:40:36.043 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:40:37.990 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:40:37.991 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:40:37.991 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:40:46.044 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:40:56.044 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:41:06.046 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:41:16.047 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:41:26.047 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:41:36.048 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:41:37.992 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:41:37.993 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:41:37.993 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:41:46.048 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:41:56.049 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:42:06.052 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:42:16.053 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:42:26.054 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:42:36.054 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:42:37.994 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:42:37.995 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:42:37.995 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:42:46.055 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:42:56.056 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:43:06.057 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:43:16.057 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:43:26.058 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:43:36.059 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:43:37.996 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:43:46.059 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:43:56.060 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:44:06.061 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:44:16.062 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:44:26.063 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:44:36.064 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:44:37.997 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:44:37.998 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:44:46.065 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:44:56.066 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:45:06.067 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:45:16.068 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:45:26.068 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:45:36.069 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:45:37.999 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:45:38.000 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:45:38.000 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:45:46.070 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:45:56.070 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:46:06.071 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:46:16.072 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:46:26.072 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:46:36.073 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:46:38.001 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:46:38.002 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:46:38.002 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:46:46.074 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:46:56.075 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:47:06.076 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:47:16.077 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:47:26.078 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:47:36.078 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:47:38.003 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:47:38.004 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:47:38.005 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:47:46.079 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:47:56.080 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:48:06.080 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:48:16.081 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:48:26.082 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:48:36.082 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:48:38.006 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:48:38.006 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:48:38.007 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:48:46.083 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:48:56.084 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:49:06.085 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:49:16.086 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:49:26.086 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:49:36.087 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:49:38.008 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:49:38.009 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:49:38.009 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:49:46.088 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:49:56.088 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:50:06.089 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:50:16.090 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:50:26.090 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:50:36.091 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:50:38.010 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:50:38.011 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:50:38.011 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:50:46.092 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:50:56.093 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:51:06.094 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:51:16.094 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:51:26.095 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:51:36.096 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:51:38.012 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:51:38.013 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:51:38.013 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:51:46.097 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:51:56.097 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:52:06.098 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:52:16.099 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:52:26.100 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:52:36.101 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:52:38.014 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:52:38.015 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:52:38.015 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:52:46.101 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:52:56.102 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:53:06.102 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:53:16.103 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:53:26.104 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:53:36.105 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:53:38.016 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:53:38.017 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:53:38.018 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:53:46.105 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:53:56.106 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:54:06.107 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:54:16.108 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:54:26.108 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:54:36.109 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:54:38.019 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:54:38.019 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:54:38.020 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:54:46.110 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:54:56.110 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:55:06.111 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:55:16.112 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:55:26.113 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:55:36.114 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:55:38.020 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:55:38.021 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:55:38.021 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:55:46.114 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:55:56.115 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:56:06.116 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:56:16.117 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:56:26.117 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:56:36.118 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:56:38.023 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:56:38.024 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:56:38.024 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:56:46.119 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:56:56.119 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:57:06.120 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:57:16.122 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:57:26.122 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:57:36.123 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:57:38.025 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:57:38.026 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:57:38.026 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:57:46.124 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:57:56.124 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:58:06.125 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:58:16.126 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:58:26.127 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:58:36.127 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:58:38.027 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:58:38.028 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:58:38.028 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:58:46.128 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:58:56.129 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:59:06.130 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:59:16.130 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:59:26.131 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:59:36.132 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:59:38.029 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:59:38.030 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 09:59:38.030 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 09:59:46.133 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 09:59:56.134 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:00:06.134 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:00:16.136 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:00:26.137 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:00:36.138 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:00:38.032 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:00:38.032 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:00:38.033 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:00:46.139 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:00:56.140 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:01:06.141 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:01:16.142 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:01:26.143 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:01:36.144 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:01:38.034 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:01:38.035 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:01:38.035 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:01:46.145 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:01:56.146 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:02:06.147 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:02:16.148 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:02:26.148 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:02:36.149 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:02:38.036 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:02:38.037 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:02:38.037 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:02:46.150 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:02:56.150 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:03:06.151 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:03:16.152 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:03:26.153 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:03:36.154 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:03:38.038 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:03:38.039 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:03:38.040 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:03:46.155 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:03:56.155 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:04:06.156 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:04:16.158 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:04:26.158 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:04:36.159 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:04:38.041 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:04:38.042 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:04:38.042 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:04:46.160 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:04:56.161 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:05:06.161 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:05:16.163 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:05:26.164 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:05:36.165 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:05:38.043 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:05:38.044 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:05:38.044 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:05:46.166 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:05:56.166 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:06:06.167 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:06:16.168 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:06:26.168 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:06:36.169 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:06:38.045 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:06:38.046 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:06:38.046 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:06:46.171 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:06:56.172 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:07:06.173 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:07:16.174 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:07:26.174 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:07:36.175 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:07:38.047 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:07:38.048 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:07:38.048 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:07:46.176 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:07:56.177 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:08:06.177 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:08:16.178 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:08:26.179 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:08:36.180 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:08:38.049 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:08:38.050 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:08:38.050 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:08:46.181 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:08:56.182 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:09:06.183 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:09:16.183 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:09:26.184 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:09:36.185 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:09:38.051 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:09:38.052 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:09:38.052 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:09:46.186 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:09:56.186 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:10:06.187 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:10:16.188 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:10:26.188 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:10:36.189 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:10:38.053 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:10:38.053 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:10:38.054 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:10:46.190 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:10:56.191 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:11:06.192 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:11:16.193 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:11:26.194 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:11:36.194 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:11:38.055 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:11:38.055 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:11:38.056 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:11:46.195 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:11:56.196 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:12:06.196 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:12:16.197 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:12:26.198 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:12:36.198 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:12:38.057 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:12:38.057 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:12:38.058 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:12:46.199 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:12:56.200 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:13:06.201 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:13:16.201 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:13:26.202 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:13:36.203 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:13:38.058 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:13:38.059 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:13:38.059 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:13:46.203 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:13:56.204 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:14:06.205 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:14:16.206 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:14:26.207 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:14:36.207 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:14:38.061 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:14:38.061 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:14:38.061 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:14:46.208 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:14:56.209 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:15:06.209 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:15:16.210 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:15:26.211 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:15:36.212 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:15:38.062 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:15:38.063 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:15:38.064 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:15:46.213 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:15:56.213 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:16:06.214 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:16:16.214 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:16:26.215 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:16:36.216 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:16:38.065 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:16:38.066 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:16:38.066 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:16:46.218 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:16:56.218 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:17:06.219 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:17:16.219 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:17:26.220 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:17:36.221 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:17:38.067 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:17:38.068 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:17:38.068 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:17:46.222 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:17:56.223 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:18:06.223 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:18:16.225 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:18:26.225 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:18:36.226 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:18:38.069 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:18:38.070 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:18:38.070 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:18:46.227 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:18:56.227 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:19:06.228 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:19:16.229 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:19:26.229 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:19:36.230 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:19:38.071 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:19:38.072 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:19:38.072 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:19:46.231 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:19:56.231 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:20:06.232 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:20:16.233 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:20:26.233 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:20:36.234 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:20:38.073 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:20:38.074 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:20:38.074 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:20:46.235 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:20:56.236 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:21:06.237 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:21:16.238 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:21:26.238 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:21:36.239 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:21:38.075 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:21:38.076 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:21:38.076 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:21:46.240 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:21:56.240 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:22:06.241 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:22:16.242 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:22:26.243 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:22:36.243 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:22:38.077 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:22:38.078 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:22:38.078 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:22:46.244 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:22:56.245 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:23:06.245 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:23:16.246 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:23:26.247 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:23:36.248 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:23:38.079 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:23:38.080 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:23:38.080 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:23:46.248 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:23:56.249 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:24:06.250 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:24:16.251 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:24:26.252 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:24:36.252 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:24:38.082 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:24:38.082 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:24:38.083 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:24:46.253 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:24:56.254 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:25:06.255 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:25:16.256 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:25:26.256 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:25:36.258 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:25:38.083 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:25:38.084 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:25:38.084 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:25:46.259 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:25:56.259 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:26:06.260 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:26:16.261 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:26:26.262 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:26:36.263 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:26:38.086 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:26:38.086 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:26:38.087 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:26:46.264 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:26:56.265 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:27:06.265 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:27:16.266 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:27:26.267 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:27:36.268 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:27:38.088 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:27:38.089 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:27:38.089 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:27:46.268 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:27:56.269 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:28:06.270 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:28:16.271 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:28:26.272 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:28:36.272 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:28:38.091 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:28:38.092 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:28:38.092 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:28:46.273 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:28:56.274 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:29:06.275 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:29:16.275 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:29:26.276 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:29:36.277 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:29:38.093 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:29:38.094 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:29:38.094 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:29:46.278 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:29:56.278 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:30:06.279 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:30:16.280 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:30:26.281 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:30:36.281 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:30:38.095 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:30:38.096 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:30:38.096 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:30:46.282 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:30:56.282 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:31:06.283 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:31:16.283 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:31:26.284 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:31:36.285 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:31:38.097 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:31:38.098 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:31:38.098 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:31:46.286 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:31:56.286 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:32:06.287 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:32:16.288 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:32:26.288 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:32:36.289 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:32:38.099 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:32:38.099 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:32:38.100 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:32:46.289 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:32:56.290 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:33:06.291 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:33:16.292 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:33:26.292 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:33:36.293 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:33:38.101 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:33:38.102 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:33:38.102 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:33:46.294 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:33:56.295 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:34:06.296 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:34:16.296 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:34:26.297 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:34:36.298 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:34:38.103 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:34:38.104 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:34:38.104 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:34:46.298 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:34:56.299 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:35:06.299 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:35:16.300 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:35:26.301 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:35:36.302 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:35:38.105 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:35:38.105 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:35:38.106 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:35:46.303 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:35:56.303 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:36:06.305 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:36:16.306 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:36:26.307 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:36:36.307 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:36:38.107 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:36:38.108 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:36:38.108 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:36:46.308 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:36:56.309 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:37:06.310 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:37:16.311 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:37:26.312 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:37:36.313 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:37:38.109 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:37:38.110 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:37:38.111 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:37:46.314 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:37:56.314 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:38:06.315 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:38:16.316 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:38:26.317 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:38:36.317 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:38:38.112 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:38:38.112 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:38:38.113 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:38:46.318 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:38:56.319 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:39:06.320 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:39:16.320 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:39:26.321 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:39:36.322 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:39:38.114 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:39:38.114 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:39:38.115 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:39:46.323 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:39:56.323 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:40:06.324 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:40:16.325 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:40:26.326 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:40:36.327 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:40:38.116 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:40:38.117 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:40:38.117 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:40:46.328 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:40:56.328 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:41:06.329 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:41:16.330 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:41:26.331 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:41:36.332 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:41:38.118 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:41:46.333 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:41:56.334 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:42:06.335 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:42:16.337 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:42:26.338 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:42:36.339 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:42:38.120 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:42:38.121 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:42:38.121 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:42:46.340 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:42:56.341 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:43:06.341 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:43:16.342 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:43:26.343 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:43:36.344 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:43:38.123 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:43:38.123 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:43:38.124 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:43:46.344 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:43:56.345 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:44:06.346 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:44:16.347 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:44:26.348 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:44:36.349 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:44:38.125 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:44:38.125 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:44:38.126 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:44:46.350 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:44:56.350 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:45:06.351 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:45:16.353 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:45:26.354 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:45:36.354 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:45:38.127 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:45:38.128 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:45:38.128 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:45:46.356 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:45:56.356 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:46:06.357 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:46:16.361 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:46:26.362 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:46:36.363 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:46:38.129 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:46:38.130 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:46:38.131 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.
>2018-06-25 10:46:46.364 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:46:56.364 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:47:06.365 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:47:16.366 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:47:26.367 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:47:36.367 298965 ERROR cinder.service [-] Manager for service cinder-volume hostgroup@tripleo_ceph is reporting problems, not sending heartbeat. Service will appear "down".
>2018-06-25 10:47:38.132 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._publish_service_capabilities run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:47:38.133 298965 DEBUG oslo_service.periodic_task [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Running periodic task VolumeManager._report_driver_status run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215
>2018-06-25 10:47:38.133 298965 WARNING cinder.volume.manager [req-dfefbefe-ebd3-43ec-8740-f4b49580a2e7 - - - - -] Update driver status failed: (config name tripleo_ceph) is uninitialized.

Actions: View

Attachments on bug 1594698: 1454319